VDAOvideo

Local AI Explained | Hardware, Setup and Models

Syntax · 383K views · 8 months ago
Description

In this video CJ guides you through the wide world of local AI. He shows how he set up his new 128GB memory mini PC and gives his overall impressions of running local models for basic tasks and coding. 00:00 - intro 01:03 - key terms / concepts 01:40 - we need gpus 02:16 - LLMs need a lot of VRAM 03:20 - model quantization 04:13 - where to find models 05:13 - GPU pricing 06:33 - unified memory PCs 10:58 - unboxing the mini pc 12:13 - fedora install / setup 13:28 - GTT memory configuration 14:04 - how to run local models 16:33 - how to download and find models 18:54 - impressions of basic prompting 20:31 - coding overall impressions 23:21 - conclusions GMKtec EVO-X2 AI Mini PC | https://amzn.to/4bbgdMg What is LLM Inference | https://huggingface.co/blog/Kseniase/ Large Language Models e