How to Run BIG AI Models with Just 8GB of VRAM
AsapGuide
0:00 / 0:00
How to Run BIG AI Models with Just 8GB of VRAM
7 170 просмотров · 2 мес. назад
AsapGuide
25,8 тыс. подписчиков
7 170 просмотров · 2 мес. назад
👉 In this video, I will show you how to run a 27 billion parameter AI model on a graphics card with only 8GB of memory using a new one bit compression method called Bonsai 27B. This local setup lets you chat, read images, and even use web search entirely offline through LM Studio, so you do not need an expensive GPU or a cloud subscription to get near flagship level performance. If you have been priced out of running big open source language models because your GPU memory is too small, this approach gets around that limit without giving up much intelligence.
https://prismml.com/news/bonsai-27b
❤️ Subscribe: / @asapguide