VDAOvideo

Local AI Explained: The Full Beginner Course (Master In 25 Minutes)

Bart Explains · 46.7K views · 3 days ago
Description

It took me about six months to properly understand how to run AI models on my own computer. This video is everything I learned, in one place. We follow one model, Llama 3.1 8B, from the file you download all the way to a working setup on a 16GB Mac mini. In the first part you'll learn what's actually inside a model file, why it has to fit in your memory, how the chat itself takes up memory, and how shrinking a model (quantization) makes it fit. In the second part we pick a file, run it in an app, and give it a harness with tools like web search. By the end you'll be able to look at your own computer, pick a model that fits, and set it up yourself. Local models won't be as good as the newest Claude or ChatGPT, but they're free, private, and run right there on your own desk. Tools from this video: - LM Studio: https://lmstudio.ai - Pi (the harness I use): https://pi.dev - Brave Search API (web search for Pi): https://brave.com/search/api/ - Ollama: https://ollama.com - llama.cpp: https://github.com/ggml-org/llama.cpp - oMLX: https://github.com/jundot/omlx - Llama 3.1 8B (GGUF): https://huggingface.co/lmstudio-community/Meta-Llama-3.1-8B-Instruct-GGUF My rule of thumb: keep the model file to about half of your memory. On a 16GB Mac, that's a file of up to about 8GB. Chapters: 0:00 Intro 1:19 Where AI models actually run 2:40 What's inside a model file 3:55 Why the model has to fit in memory 4:55 Your chat needs memory too (context) 6:04 Memory size vs memory speed 6:59 Machines people buy for AI 8:18 Writing speed vs reading speed 9:25 Why bigger models need more memory 10:05 Quantization: shrinking a model 11:59 Why small models get confused 13:06 Part 2: GGUF vs MLX 14:17 The engine and the app 15:16 The harness: how a model gets tools 16:43 LM Studio and Pi as harnesses 18:14 Step 1: Check your memory 18:38 Step 2: Pick a model that fits 19:39 Step 3: Download it 20:24 Step 4: Your first chat 20:48 Step 5: Give it a harness (Pi + web search) 22:18 When something goes wrong 22:46 Which app to use: LM Studio, Ollama, llama.cpp, oMLX 23:45 How big a model can you run? 24:15 Summary #localai #lmstudio #llama