Home / Learn / Courses / Offline AI / Cheat sheet
All lessons

Offline AI cheat sheet

The commands from the course on one page. Model names and paths are examples: use your own. On Windows PowerShell, set a variable with $env:OLLAMA_HOST="0.0.0.0"; on Mac and Linux, export OLLAMA_HOST=0.0.0.0.

Ollama Lesson 9.2

ollama pull gemma4:12bDownload a model
ollama run gemma4:12bChat in the terminal (type /bye to leave)
ollama listModels on disk
ollama psModels loaded now, their size, and GPU or CPU
ollama stop gemma4:12bUnload a model from memory
ollama rm gemma4:12bDelete a model from disk
ollama show gemma4:12bContext length, quantisation and licence
ollama create mybot -f ModelfileSave a model with your own system prompt and settings

Ollama settings (environment variables) Lessons 11.1 and 15.2

OLLAMA_HOST=0.0.0.0Listen on the network, not only this computer (inside a trusted network only)
OLLAMA_NUM_PARALLEL=4Answer up to 4 requests at once
OLLAMA_CONTEXT_LENGTH=8192Default context length
OLLAMA_KV_CACHE_TYPE=q8_08-bit KV cache to save memory (needs flash attention on)
OLLAMA_KEEP_ALIVE=30mHow long a model stays loaded after use
OLLAMA_MODELS=D:\modelsWhere model files are stored

LM Studio command line Lesson 9.1

lms lsModels on disk
lms load <model>Load a model
lms server startStart the local API server (port 1234)
lms psModels loaded now

llama.cpp Lesson 9.3

llama-cli -m model.gguf -p "Hello"One prompt in the terminal
llama-server -m model.gguf -c 8192 -ngl 99 --port 8080API server: 8,192-token context, all layers on the GPU
llama-server -m model.gguf -c 32768 -np 44 parallel slots sharing the context
--cache-type-k q8_0 --cache-type-v q8_08-bit KV cache
--split-mode layer --tensor-split 3,2Split a model across two cards
llama-bench -m model.ggufMeasure prefill and writing speed

MLX on a Mac Lesson 9.4

pip install mlx-lmInstall
mlx_lm.generate --model <model> --prompt "Hello"One prompt
mlx_lm.server --model <model>API server (port 8080)
mlx_lm.lora --model <model> --train --data ./dataTrain a LoRA adapter

Serving many users Lesson 15.2

vllm serve <model> --max-model-len 16384vLLM API server (Linux, NVIDIA)
vllm serve <model> --tensor-parallel-size 2Split across 2 cards
python -m sglang.launch_server --model-path <model>SGLang API server

Test the API Lesson 11.1

curl http://localhost:11434/api/tagsIs Ollama running? Lists models
curl http://localhost:11434/v1/modelsThe same, OpenAI-style
curl http://localhost:8080/healthIs llama-server ready?

Watch the machine Lesson 15.5

nvidia-smi -l 2NVIDIA memory, load, heat and power every 2 seconds
curl http://localhost:8080/metricsllama-server counters (start it with --metrics)
Activity Monitor > MemoryMac memory pressure (keep it green)
Task Manager > Performance > GPUWindows graphics memory and load

Check a download Lesson 7.6

sha256sum model.ggufLinux
shasum -a 256 model.ggufMac
certutil -hashfile model.gguf SHA256Windows

Python: ask a local model Lesson 11.2

from openai import OpenAI

client = OpenAI(base_url="http://localhost:11434/v1", api_key="local")
reply = client.chat.completions.create(
    model="gemma4:12b",
    messages=[{"role": "user", "content": "Say hello in five words."}],
)
print(reply.choices[0].message.content)

Install the library once with pip install openai. To use another engine, change the address: LM Studio http://localhost:1234/v1, llama-server and MLX http://localhost:8080/v1, vLLM http://localhost:8000/v1.

The offline test Lesson 9.6

  1. Load your model and front end.
  2. Turn off Wi-Fi and unplug the cable.
  3. Ask a question and upload a document.
  4. Everything still works: you are offline.

See also the glossary. Commands checked against each tool's documentation in November 2026; flags change between versions, so run a tool with --help if one is refused.