Home / Learn / Data & AI / Offline AI
All lessons

Offline AI: run AI models on your own computer

Download AI models and run them on your own laptop, desktop or office server, with no internet and no monthly limit. From what a token is to building private document search, with prices in India, Canada, the USA and the UK.

For adults, students and working professionals. No coding needed to start; later modules use a little Python. Free, no sign-up. All 119 lessons, about 43.8 hours.

What you will have at the end

  • A private assistant that runs on your own laptop or desktop with the internet switched off.
  • Search across your own documents: ask questions of your PDFs and notes and get answers with the page they came from.
  • Your own workflows: small Python programs that send work to the model on your machine, such as summarising a folder of files every night.
  • A cost sheet comparing your setup with Claude, ChatGPT and Google plans, in rupees, Canadian dollars, US dollars or pounds.

Want to know how the model works inside, with the maths? The free course From a straight line to an LLM builds up from linear regression to attention and transformers, with a simulation in every lesson.

Quick reference: glossary of terms · cheat sheet of commands

0 of 119 lessons done

Without the tick, your progress lasts until you close this tab. Either way it stays in this browser: there is no sign-up and nothing is sent to us.

All modules

Part A: Foundations

1 Welcome, and why run AI yourself 5 lessons

  1. 1.1 What this course gives you
  2. 1.2 Cloud AI and local AI: the same brain in a different house
  3. 1.3 What "offline" really means
  4. 1.4 Pick your track: laptop, desktop or team server
  5. 1.5 Five myths about running AI at home

2 How a language model works 7 lessons

  1. 2.1 Text in, text out: predicting the next token
  2. 2.2 Parameters: what "7B" and "120B" mean
  3. 2.3 Training and inference: learning versus using
  4. 2.4 Transformers and attention, explained with a classroom
  5. 2.5 Dense models and Mixture-of-Experts
  6. 2.6 Base, instruct, reasoning and multimodal models
  7. 2.7 Why models make things up

3 Tokens: the unit of everything 10 lessons

  1. 3.1 What a token is
  2. 3.2 Tokenizers differ: languages, code and counts
  3. 3.3 The context window: a model's working memory
  4. 3.4 Input, output and thinking tokens
  5. 3.5 Tokens per second: reading speed and writing speed
  6. 3.6 Saving tokens 1: short, clear prompts
  7. 3.7 Saving tokens 2: trimming and summarising context
  8. 3.8 Saving tokens 3: caching, limits and structured output
  9. 3.9 Why your own machine makes tokens "unlimited"
  10. 3.10 Measure tokens and speed on your own machine

Part B: Hardware

4 The parts that matter for AI 6 lessons

  1. 4.1 Memory is king: the model must fit
  2. 4.2 Memory bandwidth: the real speed limit
  3. 4.3 GPU, CPU, NPU and unified memory
  4. 4.4 Storage, RAM, power supply and cooling
  5. 4.5 Laptops, desktops, Macs, mini PCs and AI boxes
  6. 4.6 AI on phones and tablets

5 Buying guide with prices in four countries 7 lessons

  1. 5.1 Budget tier: use what you have
  2. 5.2 Starter tier: a 16 GB graphics card or a 32 GB Mac
  3. 5.3 Enthusiast tier: 24 to 128 GB
  4. 5.4 128 GB AI boxes
  5. 5.5 A server for a team or office
  6. 5.6 New or used, import or local, warranty and tax
  7. 5.7 The 2026 memory shortage and how to buy well

6 Power, heat and running costs 4 lessons

  1. 6.1 Watts at rest and at full load
  2. 6.2 What an hour of AI costs in electricity
  3. 6.3 Cooling, noise and where to put the machine
  4. 6.4 Laptop batteries and thermal throttling

Part C: Software and setup

7 Getting and choosing models 6 lessons

  1. 7.1 Where models live
  2. 7.2 Open weights, open source and reading a licence
  3. 7.3 Model families today and what each is good at
  4. 7.4 Model cards, benchmarks and why leaderboards mislead
  5. 7.5 File formats: GGUF, safetensors and MLX
  6. 7.6 Safe downloads and storage planning

8 Quantisation: shrinking models to fit 5 lessons

  1. 8.1 What quantisation is
  2. 8.2 Bits per weight and what you lose
  3. 8.3 Common formats: Q4_K_M, AWQ, MLX and FP4
  4. 8.4 The KV cache: the hidden memory cost of long context
  5. 8.5 Choosing the right size for your machine

9 Installing your first local AI 7 lessons

  1. 9.1 The easiest path: a desktop app
  2. 9.2 Ollama: models from the terminal
  3. 9.3 llama.cpp: the engine underneath
  4. 9.4 MLX on a Mac
  5. 9.5 A chat window that runs on your machine
  6. 9.6 Pull the cable: testing it fully offline
  7. 9.7 Fixing the common problems

10 Settings and prompting 5 lessons

  1. 10.1 Temperature, top-p and the other dials
  2. 10.2 System prompts and chat templates
  3. 10.3 Context length settings and their memory cost
  4. 10.4 Reasoning models at home
  5. 10.5 Prompts that work well on smaller models

Part D: Building offline pipelines

11 The local API 4 lessons

  1. 11.1 Your computer as an AI server
  2. 11.2 Calling it from Python and scripts
  3. 11.3 Streaming, JSON output and tool calling
  4. 11.4 Switching between local and cloud with one setting

12 Private document search (RAG), fully offline 7 lessons

  1. 12.1 Why the model cannot read your files by itself
  2. 12.2 Embeddings: turning meaning into coordinates
  3. 12.3 Cutting documents into chunks
  4. 12.4 Vector databases you can run locally
  5. 12.5 Finding, re-ranking and citing sources
  6. 12.6 Reading PDFs, scans, spreadsheets and email
  7. 12.7 Testing answers: did it find the right page?

13 Agents and automation offline 8 lessons

  1. 13.1 What an agent is
  2. 13.2 Letting a local model use tools safely
  3. 13.3 A coding assistant that runs on your machine
  4. 13.4 Offline speech to text and text to speech
  5. 13.5 Reading images and screenshots locally
  6. 13.6 Making images offline
  7. 13.7 Scheduled jobs: summaries and reports overnight
  8. 13.8 Guard rails: stopping an agent doing damage

14 Customising models 5 lessons

  1. 14.1 Prompting, RAG or fine-tuning: which one you need
  2. 14.2 LoRA and QLoRA in plain words
  3. 14.3 What fine-tuning takes on home hardware
  4. 14.4 Building a small training set and checking the result
  5. 14.5 Merging and sharing your adapter

Part E: Scale, money, limits and careers

15 Scaling up: from one laptop to a team server 7 lessons

  1. 15.1 Many users, one model: batching
  2. 15.2 Serving engines built for many users
  3. 15.3 Splitting a model across graphics cards or machines
  4. 15.4 Running it on your office network
  5. 15.5 Watching speed, memory and errors
  6. 15.6 Hybrid: local first, cloud for the hardest jobs
  7. 15.7 Renting graphics cards by the hour

16 Cost: local AI against subscriptions 6 lessons

  1. 16.1 What Claude, ChatGPT and Google plans give you
  2. 16.2 Pay-per-token APIs explained
  3. 16.3 The full cost of owning the hardware
  4. 16.4 Break-even: when local becomes cheaper
  5. 16.5 When the cloud wins and when local wins
  6. 16.6 The hybrid budget most people should use

17 Pros, cons and limits 8 lessons

  1. 17.1 The quality gap: where cloud models still lead
  2. 17.2 Speed limits and where they come from
  3. 17.3 Memory limits: model, context and cache
  4. 17.4 Limits in theory: knowledge cut-off, reasoning, long context
  5. 17.5 Limits in practice: setup, updates, heat and noise
  6. 17.6 Security and privacy: what local protects and what it does not
  7. 17.7 Licences, law and sector rules in four countries
  8. 17.8 Every cloud feature and its local equivalent

18 Who uses this: sectors and careers 12 lessons

  1. 18.1 Healthcare and clinics
  2. 18.2 Law and accounting
  3. 18.3 Finance and banking
  4. 18.4 Teachers, students and offline classrooms
  5. 18.5 Software and data teams
  6. 18.6 Research and science labs
  7. 18.7 Government and air-gapped sites
  8. 18.8 Small business, retail, hotels and logistics
  9. 18.9 Field work with no internet
  10. 18.10 Writers, creators and translators
  11. 18.11 Local AI skills in job ads
  12. 18.12 Capstone: an offline AI setup for one sector