How to switch to local AI (Open-Source Claude)

I've been using LLMs locally for a few years now, ever since Llama 2. Honestly it's my favorite hobby ever and it's been fun to see the evolution of local models, and I think for the first time, we're getting to the point where local AI / local llms can replace ChatGPT and Claude. This could be a HUGE video but hopefully I conveyed the most useful information! Phew. I didn't deep dive, but basically you're going to be running .GGUF quant files of models or MLX depending on your choices here. In this video I go over using Unsloth Desktop, but you can use like LM Studio or llama.cpp directly. I go back and forth, but day-to-day I'm using oMLX + Qwen 3.8 27b with lightning MTP on and get ~15-21tk/s. This is extremely usable. 🔍 How to Run Models 1. Basically it all depends on how much VRAM you have. If you're on a Mac 16gb is probably the bare minimum. Aim for a 12b model. If you're looking at .ggufs (e.g via Unsloth) you can choose the quant. 2. Quants are labeled Q1, Q2, Q3, Q4 etc. The bigger the better, but a good standard of thumb is start with Q4_K_M. 📚 Resources Huggingface - this is where all models get uploaded to current and where Unsloth is populating from. If you can't find a specific model /r/LocalLlama - Reddit! A great helpful community for running local LLMs. More technical. /r/LocalLLM - Another great subreddit ❌ What I didn't cover Finetuning: lets you fine-tune your own model if that's your thing. If you're a company, this might be more relevant to you, training your own chatbot or whatever. oMLX / quants / MTP: There are a lot of bleeding edge updates regularly. If your performance sucks, researching like lightning MTP / Dflash can help, but it can get complicated. UI's like oMLX do make this easier though. 🙋‍♂️ Help If you need help, feel free to reach out! If you're a company as well, running LLMs on a private local server is a thing and can cut Claude / Codex costs. --- Chapters: 0:00 - Introduction 1:09 - How is it possible to run AI locally? 2:56 - The landscape of open-source models 3:57 - How to run a model (Unsloth, Llama.cpp) 4:51 - Using Unsloth 6:21 - Chatting 7:33 - Web Search 9:24 - Replacing Claude Code (Opencode) 10:45 - Setting up Opencode 12:39 - Can local AI replace ChatGPT or Claude? Tools Used: MBP 16" M3 Max 96gb Figma Codex, Zed, VsCode etc Huggingface, Unsloth Desktop, LM Studio, oMLX FCP / After Effects / Figma / Photoshop ---- Final Note: AI, as a designer, is moving so fast it can be scary and overwhelming. The design industry is definitely going to change forever. Don't feel like your falling behind; just take a deep breath, learn what you can, and do your best. You got this!