Getting Started with LOCAL AI on UNRAID - Open WebUI + Ollama

There are a ton of reasons why someone may want to start using AI on a local device. It might be to increase data privacy and security, avoid ads and complicated payment structures, or simply to learn new high value skills. Regardless of why you want to start doing it, this video walks through the easiest way to get started with Local AI on your Unraid server. It covers both a Docker Compose route and a more traditional Unraid route using the community app store. This setup uses Ollama as the back-end inference engine to find, manage, download, and actually run models. Open WebUI acts as the front end to give a user friendly platform for chatting and interacting with models. Together they can provide a shockingly robust user experience with local AI. Learn more about them here: https://ollama.com/ https://docs.openwebui.com/ Extra Parameters Commands: --gpus=all (Use in Ollama container's “Extra Parameters” to allow GPU access) --restart unless-stopped (Will restart the container unless manually stopped) Terminal Commands: nvidia-smi (Use in Ollama container to verify GPU functionality) openssl rand -base64 48 (Use to create the OPENWEBUI_SECRET_KEY variable) Extra Resources: Docker Compose Github: https://github.com/techmerc-yt/docker... Intel GPU Resources: https://docs.ollama.com/gpu#vulkan-gp... AMD GPU Resources: https://docs.ollama.com/gpu#amd-radeon Ollama CLI Guide: https://docs.ollama.com/cli 0:00 - 0:57 Introduction (Why self-host) 0:57 - 1:02 Why I Host LLMs 1:02 - 2:26 What is the Stack? 2:26 - 3:55 What is Ollama and Open WebUI? 3:55 - 4:38 Important Notes On GPUS 4:38 - 6:27 Installing Nvidia Driver Plugin 6:27 - 12:24 Docker Compose Setup 12:24 - 16:59 Unraid Setup 16:59 - 19:36 Getting Models and Testing Open WebUI 19:36 - 20:28 Outro (Like & Subscribe!!) Transcendence by Alexander Nakarada | https://creatorchords.com Music promoted by https://www.chosic.com/free-music/all/ Creative Commons CC BY 4.0 https://creativecommons.org/licenses/...