VDAOvideo

The HARD Truth About Hosting Your Own LLMs

Cole Medin · 73.8K views · 2 years ago
Description

Hosting your own LLMs like Llama 3.1 requires INSANELY good hardware - often times making running your own LLMs completely unrealistic. But I have a strategy that I reveal in this video for how to start cheap with self-hostable LLMs and then continue on to scale with them infinitely as your app/business grows... 00:00 - 02:58 - The Problem with Local LLMs 02:59 - 03:35 - The Strategy for Local LLMs 03:36 - 08:02 - Exploring Groq's Amazingness 08:03 - 13:59 - The Groq to Local LLM Quick Maths 14:00 - 14:43 - Outro ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Services I mentioned in this video (I am not sponsored by any of them): Groq: https://groq.com/ RunPod: https://www.runpod.io/ DigitalOcean: https://www.digitalocean.com/ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Artificial Intelligence is no