00000000 | AI Terminology | segfault.segfault.digital

Terminology is always absolutely important, especially in the case of "Artificial Intelligence" as it introduced within a short timespan a lot of new words.

Fyi I have used as a starting base ik_llama.cpp's "LLM Jargon" table., but I have then heavily edited it -> if you think that something is wrong or else you can send me a message by clicking on the "Contact"-link located in the navigation menu of this site :o)

Disclaimer:

I'm currently just a "user" of "llama.cpp" and similar forks, therefore I have a decent grasp of "Prompt Processing", "Token Generation", and various optimizations for "running" such an AI engine, but I have never programmed anything directly related to them.
I do fully understand how "Backpropagation Neural Networks" (BPNN) work and could program such a thing in any programming language.
Therefore my explanations about e.g. "Tensor"/"Inference"/"Layer" are purely theoretical, have a bias toward my BPNN background, and were just discussed (and agreed) with an AI (currently Google Gemini), but they might still be wrong
-> please always doublecheck what you're reading here, and if you feel like complaining then you can do so by clicking on the "Contact"-link in the navigation menu.

Cheers :o)

Agent harness	TERM PENDING TO BE REVIEWED! The tools and the infrastructure around the LLM in an AI Agent. AI Agent = LLM + Agent harness
AI	Artificial Intelligence
AI agent	TERM PENDING TO BE REVIEWED! Tool/program that uses LLM to achieve a goal/task via a series of planning/steps/actions/tool-calling/etc. Coding agents are specialized in software goals.
BPW	Bits Per Weight used by the language model. The more "bits" are used the more precise the calculations and response quality will be, but the higher the demands on the hardware needed to run it.
FA	Flash Attention is a method to improve the efficiency of both the training and the response ("tokens") generation tasks of an LLM: it improves performance by lowering the demand on specific hardware components without having any impact on the quality of the results.
GGUF	TERM PENDING TO BE REVIEWED! The file format used by "llama.cpp" and its forks (e.g. "ik_llama.cpp", the one of "PrismML", etc...).
imatrix	An "Importance MATRIX" is a calibration file used during model compression (quantization), which analyzes sample text to identify which internal parameters ("weights") are most critical to output quality, ensuring those key weights are preserved with higher accuracy when squeezing the model into lower bit sizes (the quantized versions of files that start with "IQ" use this method).
Inference	The process used by an AI model to 1) "digest" your query ("prompt processing"/"pp") and 2) generate its response (creating the single "tokens" that become the output being text/image/audio/whatever that you get).
Layer	A logical layout of computations, the flow moving from an "input"-stage to the final "output"-stage; usually (99.999%) this also includes multiple additional internal stages (known in classical "backpropagation neural networks" as "hidden" layers).
LLM	TERM PENDING TO BE REVIEWED! Large Language Model - a language model trained on a vast amount of text.
MCP	TERM PENDING TO BE REVIEWED! Model Context Protocol, an open standard for the way artificial intelligence (AI) systems like large language models (LLMs) integrate and share data with external tools, systems, and data sources
model splits	TERM PENDING TO BE REVIEWED! GGUF file can be split in multiple parts to simplify upload/download. When using such model, specify only the first part.
PP	TERM PENDING TO BE REVIEWED! Prompt Processing.
quants	TERM PENDING TO BE REVIEWED! The "compressed" format of the model.
RAG	TERM PENDING TO BE REVIEWED! Retrieval Augmented Generation. Provide external documents to the LLM for information lookup.
t/s	TERM PENDING TO BE REVIEWED! Token/second, measures PP and TG.
Tensors	PARTIAL ~REVIEW PENDING! Just an array of integers/floats/booleans, usually multidimensional, organized in a predefined structure, using the currently superfancy name "Tensor" which in turn refers to a pre-structured array. If you ever created a variable in any language like for example in the "C" programming language... `int my_variable = 132;` ...then you would create a "tensor" in Python using the PyTorch-module with... `my_tensor = torch.tensor([132])` ...or if you would do the same for a 2x2 multidimensional array then you would use... `my_matrix_tensor = torch.tensor([ [1.0, 2.0], [4.0, 5.0] ])` ...so nothing magic here -> the word "tensor" is just a new word to describe potentially multidimensional variables packaged to be well understood by AI engines. If you are already a programmer there is nothing new to learn: whenever you hear "tensor" then think about a "potentially multidimensional variable/array".
TG	TERM PENDING TO BE REVIEWED! Token generation.
VRAM	TERM PENDING TO BE REVIEWED! Dedicated memory in GPU.
Weights	TERM PENDING TO BE REVIEWED! Numerical values associated with the connections between tensors in the layers.