All lessons
Offline AI glossary
147 words from the course, in plain English. "Also called" lists the other names you will meet in apps, papers and job ads. Each term links to the lesson that explains it.
No term matches. Try a shorter word.
A
- Active parameters
- Also called: active weights. The share of a mixture-of-experts model used for each token. Sets speed, while total parameters set memory. Lesson 2.5
- Adapter
- Also called: LoRA adapter. A small file of extra weights that changes a model's habits without changing the base model file. Lesson 14.2
- Agent
- Also called: AI agent, agentic AI. A model that works in a loop: decides a step, uses a tool, reads the result, and repeats until the task is done. Lesson 13.1
- Air gap
- Also called: air-gapped network, isolated network. A network with no connection to the internet; software and data cross only on checked media. Lesson 18.7
- Apache 2.0
- Also called: permissive licence. A licence that allows use, changes and selling, as long as you keep the notice. Used by Gemma 4, gpt-oss and others. Lesson 7.2
- API
- Also called: application programming interface, endpoint. A way for programs to talk to each other. Local engines offer an API so your scripts can send prompts. Lesson 11.1
- API key
- Also called: secret key, token (in the security sense). A password for a paid API. Keep it in an environment variable, never in shared code. Lesson 11.4
- Attention
- Also called: self-attention. The step where each token looks at earlier tokens to decide what matters for the next word. Lesson 2.4
- Audit log
- Also called: access log. A record of who used a system and when, kept to check that rules are followed. Lesson 18.1
- AWQ
- Also called: activation-aware quantisation. A 4-bit model format used mainly by vLLM on graphics cards. Lesson 15.2
B
- Bandwidth
- Also called: memory bandwidth, GB/s. How fast data moves between memory and the chip. Sets the top writing speed of a model. Lesson 4.2
- Base model
- Also called: pretrained model, foundation model. A model trained only to continue text, before it is taught to follow instructions. Lesson 2.6
- Batch processing
- Also called: batch API. Sending many jobs to be done later, often at half price on cloud APIs. Lesson 16.2
- Batching
- Also called: concurrent requests, parallel decoding. Writing several answers in one pass over the weights, so one machine serves many people. Lesson 15.1
- Benchmark
- Also called: eval, leaderboard score. A standard test used to compare models. Useful, but your own tasks matter more. Lesson 7.4
- Bits per weight
- Also called: bpw, precision. How many bits store each parameter. Fewer bits means a smaller, faster model with some quality loss. Lesson 8.2
- Blind test
- Also called: blind comparison. Comparing answers without knowing which model wrote which. Lesson 17.1
- Break-even
- Also called: payback period. The month when owning hardware has cost less in total than the cloud plan it replaces. Lesson 16.4
C
- Cached input
- Also called: prompt caching, prefix caching. A repeated start of a prompt kept ready, so it costs less and is read faster next time. Lesson 3.8
- Checksum
- Also called: hash, SHA-256. A short fingerprint of a file. If it matches the publisher's, the file is unchanged. Lesson 7.6
- Chunk
- Also called: passage, text chunk. A piece of a document, a few hundred tokens long, stored for search. Lesson 12.3
- Chunking
- Also called: text splitting. Cutting documents into chunks before making embeddings. Lesson 12.3
- Citation
- Also called: source reference. The document and page an answer came from, so a person can check it. Lesson 12.5
- Cloud AI
- Also called: hosted AI, SaaS AI. AI that runs on a company's servers; your words travel there and back. Lesson 1.2
- Cluster
- Also called: multi-node setup. Several computers working together to run one model. Lesson 15.3
- Coder model
- Also called: code model. A model trained mostly on code, for programming help. Lesson 13.3
- Compute-bound
- Also called: limited by arithmetic. A step whose speed is set by how many sums the chip can do, such as reading a long prompt. Lesson 17.2
- Context length
- Also called: context window, context size, num_ctx. How many tokens the model can see at once: your prompt, the chat so far and its answer. Lesson 3.3
- Continuous batching
- Also called: in-flight batching. Batching where requests join and leave at every step, so nobody waits for the slowest. Lesson 15.1
- CUDA
- Also called: NVIDIA compute platform. NVIDIA's software layer that lets programs use its graphics cards for maths. Lesson 4.3
D
- Data minimisation
- Also called: need to know. Using only the personal data a job needs, and keeping it only as long as needed. Lesson 17.7
- Decode
- Also called: generation, token generation. Writing the answer one token at a time. Limited by memory bandwidth. Lesson 17.2
- Dense model
- Also called: dense transformer. A model that uses all its parameters for every token. Lesson 2.5
- Distillation
- Also called: knowledge distillation. Training a small model to copy a larger model's answers. Lesson 2.3
E
- Effective context length
- Also called: usable context. The length a model actually handles well, often shorter than the advertised maximum. Lesson 17.4
- Embedding
- Also called: vector, text embedding. A list of numbers that stands for the meaning of a text, so similar texts get similar numbers. Lesson 12.2
- Embedding model
- Also called: encoder. A small model that turns text into embeddings for search. Lesson 12.2
- Engine
- Also called: inference engine, runtime, backend. The program that loads a model and runs it: Ollama, LM Studio, llama.cpp, vLLM, MLX. Lesson 9.2
- Environment variable
- Also called: env var. A setting stored outside your code, such as OLLAMA_HOST or an API key. Lesson 11.4
- Expert
- Also called: MoE expert. One of the sub-networks in a mixture-of-experts model; a router picks a few per token. Lesson 2.5
F
- Fine-tuning
- Also called: post-training, supervised fine-tuning, SFT. Training an existing model a little more on your examples to change its style or skills. Lesson 14.1
- Firewall
- Also called: network filter. A rule set that decides which computers may reach which ports. Lesson 15.4
- FP16
- Also called: half precision, 16-bit. Numbers stored in 16 bits; the usual size of unquantised model weights. Lesson 8.2
- Front end
- Also called: chat UI, interface. The window you type into, such as Open WebUI, separate from the engine that runs the model. Lesson 9.5
- Frontier model
- Also called: state-of-the-art model. The strongest models of the moment, usually cloud-only. Lesson 17.1
- Function calling
- Also called: tool calling, tool use. A model replying with a structured request to run a tool, which your program then runs. Lesson 13.2
G
- GGUF
- Also called: llama.cpp model file. The single-file model format used by llama.cpp, Ollama and LM Studio. Lesson 7.5
- GPTQ
- Also called: GPT quantisation. A 4-bit format for graphics cards, used with vLLM and similar engines. Lesson 8.3
- GPU
- Also called: graphics card, video card, accelerator. A chip with thousands of small cores and fast memory; the usual home for local models. Lesson 4.3
- Grouped-query attention
- Also called: GQA. A design that shares key and value heads, making the KV cache smaller. Lesson 8.4
- Guard rails
- Also called: safeguards, approvals. Limits on what an agent may do, such as asking a person before sending or deleting. Lesson 13.8
H
- Hallucination
- Also called: confabulation, made-up facts. A confident answer that is wrong or invented. Lesson 2.7
- Head
- Also called: attention head. One of several parallel attention units in each layer. Lesson 2.4
- Hugging Face
- Also called: model hub. The main website for downloading open models and datasets. Lesson 7.1
- Hybrid
- Also called: hybrid AI, local-first. Using local AI by default and the cloud only for jobs that need it and may leave. Lesson 15.6
I
- Inference
- Also called: running a model, serving. Using a trained model to produce answers. What this course does. Lesson 2.3
- Input tokens
- Also called: prompt tokens. The tokens you send: instructions, chat history, documents. Lesson 3.4
- Instruct model
- Also called: chat model, instruction-tuned model. A model trained to follow instructions and chat. The usual choice. Lesson 2.6
J
- JSON mode
- Also called: structured output. Asking the model to reply in strict JSON so a program can read it. Lesson 11.3
K
- Knowledge cut-off
- Also called: training cut-off. The date the model's training data ends; it knows nothing after. Lesson 17.4
- KV cache
- Also called: key-value cache, attention cache. Memory that stores each token's keys and values so they need not be recomputed. Grows with context. Lesson 8.4
- KV cache quantisation
- Also called: q8_0 cache, cache type. Storing the KV cache in 8 or 4 bits to save memory. Lesson 17.3
L
- Latency
- Also called: delay, response time. How long you wait. For AI, often split into time to first token and writing time. Lesson 3.5
- Layer
- Also called: transformer block. One stage of the model; tokens pass through dozens of layers in order. Lesson 2.4
- Least privilege
- Also called: least access, least power. Giving each tool or person only the access a job needs. Lesson 17.6
- Licence
- Also called: model licence, terms of use. The rules for using a model: commercial use, limits, notices. Lesson 7.2
- llama.cpp
- Also called: llama-server, llama-cli. Open-source engine that runs GGUF models on almost any hardware. Lesson 9.3
- LM Studio
- Also called: desktop app. A free desktop app for downloading and chatting with local models. Lesson 9.1
- LoRA
- Also called: low-rank adaptation. A cheap way to fine-tune by training a small adapter instead of the whole model. Lesson 14.2
- Lost in the middle
- Also called: middle-context drop. Models use the start and end of a long input better than the middle. Lesson 17.4
M
- Memory shortage
- Also called: DRAM shortage, HBM squeeze. The 2026 squeeze on memory chips that raised graphics card and computer prices. Lesson 5.7
- Memory-bound
- Also called: limited by bandwidth. A step whose speed is set by how fast memory delivers data, such as writing tokens. Lesson 17.2
- Metrics
- Also called: telemetry counters, /metrics. Numbers a server reports about itself: requests, speed, errors. Lesson 15.5
- Mixture of experts
- Also called: MoE, sparse model. A model split into experts, using only a few for each token: big knowledge, faster speed. Lesson 2.5
- MLX
- Also called: Apple MLX. Apple's framework for running and fine-tuning models fast on Apple silicon. Lesson 9.4
- Model card
- Also called: model page, README. The page that describes a model: who made it, what it is for, licence, limits. Lesson 7.1
- Model family
- Also called: model series. Related models from one maker in several sizes, such as Gemma 4 or Qwen. Lesson 7.3
- Model risk management
- Also called: MRM, model governance. Testing, documenting and monitoring models used in decisions, as finance regulators expect. Lesson 18.3
- Multimodal model
- Also called: vision-language model, VLM. A model that reads images (and sometimes audio) as well as text. Lesson 13.5
N
- Next-token prediction
- Also called: autoregressive generation. How language models write: predict one token, add it, predict the next. Lesson 2.1
- NPU
- Also called: neural processing unit, AI accelerator. A chip part built for AI maths in many new laptops and phones; small models only, for now. Lesson 4.3
O
- Offline
- Also called: local, on-device, air-gapped. Working with no internet: the model, engine and data are all on your machine. Lesson 1.3
- Offloading
- Also called: partial GPU offload, -ngl. Keeping some layers in normal memory when the graphics card is too small. Fits, but slower. Lesson 17.3
- Ollama
- Also called: ollama serve, ollama run. A free engine that downloads and runs models with one command and offers an API. Lesson 9.2
- On-premise
- Also called: on-prem, self-hosted, private AI. AI running on an organisation's own hardware rather than a provider's cloud. Lesson 18.11
- Open WebUI
- Also called: self-hosted chat interface. A free chat website you run yourself, with accounts, history and document upload. Lesson 9.5
- Open weights
- Also called: open model, downloadable model. A model whose trained weights you may download and run; not always fully open source. Lesson 7.2
- OpenAI-compatible API
- Also called: chat completions API. The common request format that local engines and many cloud services speak. Lesson 11.1
- Output tokens
- Also called: completion tokens. The tokens the model writes, including thinking. Usually the dearest part of an API bill. Lesson 3.4
P
- PagedAttention
- Also called: paged KV cache. vLLM's way of storing KV caches in small pages so many chats fit in memory. Lesson 15.2
- Parameters
- Also called: weights, model size, B. The numbers learned in training. "12B" means 12 billion of them. Lesson 2.2
- Pickle
- Also called: .bin, .pt, .pth files. An old Python file format for models that can run hidden code when loaded. Avoid from unknown sources. Lesson 17.6
- Pipeline split
- Also called: layer split, pipeline parallel. Putting different layers on different cards; adds memory, not speed for one user. Lesson 15.3
- Prefill
- Also called: prompt processing, prompt eval. Reading the whole prompt in one parallel pass before writing starts. Sets time to first token. Lesson 17.2
- Privacy impact assessment
- Also called: PIA, DPIA. A written check of the privacy risks of a new system, required in some cases. Lesson 17.7
- Prompt
- Also called: input, query. What you send the model: your question, instructions and any text. Lesson 10.5
- Prompt injection
- Also called: indirect prompt injection. Text inside data (a web page, email or file) that tries to give the model orders. Lesson 17.6
Q
- Quant type
- Also called: Q4_K_M, Q8_0, IQ4_XS. The name of a quantisation recipe; Q4_K_M is a common good balance. Lesson 8.3
- Quantisation
- Also called: quantization, quant, compression. Storing weights in fewer bits so a model is smaller and faster. Lesson 8.1
- Queue
- Also called: request queue. Requests waiting for a busy server to start them. Lesson 15.1
R
- RAG
- Also called: retrieval-augmented generation, document Q&A. Finding relevant passages in your documents and giving them to the model with the question. Lesson 12.1
- Rate limit
- Also called: usage limit, quota. A cap on how much you may use a service in a period. Lesson 16.1
- Reasoning model
- Also called: thinking model. A model that writes out its thinking before answering; better on hard problems, slower. Lesson 10.4
- Redaction
- Also called: masking, anonymisation. Removing names and identifying details before text is shared. Lesson 15.6
- Reproducibility
- Also called: repeatability. Being able to rerun a result later and get the same answer, by recording model, settings and prompt. Lesson 18.6
- Reranker
- Also called: cross-encoder. A model that re-sorts search results by how well they answer the question. Lesson 12.5
- Resale value
- Also called: residual value. What you can sell hardware for later; lowers the real cost. Lesson 16.3
- Retrieval
- Also called: semantic search, vector search. Finding the passages whose embeddings are closest to the question's. Lesson 12.5
- Reverse proxy
- Also called: Caddy, nginx. A small web server in front of an app that adds HTTPS and controls access. Lesson 15.4
- Router
- Also called: model router. Code that decides which model or service each request goes to. Lesson 15.6
- RPC
- Also called: remote procedure call. One program asking another over the network to do work; llama.cpp uses it to spread a model over machines. Lesson 15.3
S
- Safetensors
- Also called: .safetensors. A safe file format for weights that holds only numbers, no code. Lesson 7.5
- Sampling
- Also called: decoding settings. How the next token is picked from the model's guesses: temperature, top-p, top-k. Lesson 10.1
- Scheduled job
- Also called: cron job, scheduled task. A script that runs by itself at set times, such as a nightly summary. Lesson 13.7
- Schema
- Also called: table layout. The names and types of a database's tables and columns. Lesson 18.5
- Seed
- Also called: random seed. A number that fixes the random choices, so the same prompt gives the same answer. Lesson 10.1
- Serving engine
- Also called: inference server. An engine built to answer many users at once, such as vLLM or SGLang. Lesson 15.2
- SGLang
- Also called: serving engine. A fast open-source serving engine that reuses shared prompt starts. Lesson 15.2
- Slot
- Also called: parallel slot, -np. One conversation a server can handle at the same time. Lesson 15.1
- Spot instance
- Also called: interruptible instance. A cheaper rented machine that the provider can take back at short notice. Lesson 15.7
- Streaming
- Also called: token streaming. Showing the answer word by word as it is written instead of all at the end. Lesson 11.3
- Subscription
- Also called: plan, seat. A monthly fee for a cloud AI service with usage limits. Lesson 16.1
- System prompt
- Also called: system message, instructions. Standing instructions given before the chat that shape every answer. Lesson 10.2
T
- Telemetry
- Also called: usage reporting. Data an app sends home about its use. Turn it off on offline machines. Lesson 18.7
- Temperature
- Also called: randomness, creativity setting. A setting for how adventurous token choices are: low for facts, higher for ideas. Lesson 10.1
- Tensor parallel
- Also called: tensor split, TP. Splitting every layer across cards so they work on each token together; needs a fast link. Lesson 15.3
- Thinking tokens
- Also called: reasoning tokens. Tokens a reasoning model writes before its answer; they cost time and, on APIs, money. Lesson 3.4
- Throttling
- Also called: thermal throttling. A chip slowing itself to stay cool. Lesson 6.4
- Time to first token
- Also called: TTFT. How long until the first word appears: waiting plus prefill. Lesson 15.5
- Token
- Also called: word piece, subword. The unit a model reads and writes: a word, part of a word or a symbol. Lesson 3.1
- Tokenizer
- Also called: tokeniser, vocabulary. The part that splits text into tokens; it differs between models. Lesson 3.2
- Tokens per second
- Also called: tok/s, throughput, generation speed. How fast a model writes. About 10 feels slow, 30 comfortable, 60 or more fast. Lesson 3.5
- Top-p
- Also called: nucleus sampling. Keep only the most likely tokens that together make up p of the chance, then pick among them. Lesson 10.1
- Total cost of ownership
- Also called: TCO. Every cost of owning a machine over its life, minus what you get back. Lesson 16.3
- Training
- Also called: pretraining. Building a model's weights from huge amounts of text; needs data-centre power. Lesson 2.3
- Training data
- Also called: dataset, fine-tuning examples. The examples a model learns from; for fine-tuning, your own prompt and answer pairs. Lesson 14.4
- Transformer
- Also called: transformer architecture. The model design behind nearly all language models, built from attention layers. Lesson 2.4
U
- Unified memory
- Also called: shared memory, UMA. One pool of memory shared by the CPU and GPU, as in Macs and 128 GB AI boxes. Lesson 5.4
- Usage window
- Also called: rolling limit, 5-hour limit. A period (often 5 hours) over which a cloud plan meters your use. Lesson 16.1
- Utilisation
- Also called: load, busy share. The share of time a server is busy; queues grow fast above about 70%. Lesson 15.1
V
- Vector database
- Also called: vector store, index. A store of embeddings that finds the closest ones to a question quickly. Lesson 12.4
- vLLM
- Also called: serving engine. A widely used open-source serving engine for many users on NVIDIA and AMD servers. Lesson 15.2
- VPN
- Also called: virtual private network, WireGuard. An encrypted tunnel that puts a remote device on the office network. Lesson 15.4
- VRAM
- Also called: video memory, GPU memory. The graphics card's own fast memory; the model must fit in it to run fast. Lesson 4.1
W
- Watt
- Also called: W, power draw. How much power a device uses at a moment. Watts × hours ÷ 1,000 = kWh on your bill. Lesson 6.1
- Weights
- Also called: parameters, model file. The learned numbers in a model, stored in the model file. Lesson 2.2
- Whisper
- Also called: whisper.cpp, speech to text. An open speech recognition model that runs offline to turn audio into text. Lesson 13.4