Docker

TL;DR Moving Ollama models to another drive requires changing the OLLAMA_MODELS environment variable and relocating your existing model files. By default, Ollama stores models in ~/.ollama/models on Linux systems, but you can point it to any directory with sufficient space. The fastest approach: stop the Ollama service, set OLLAMA_MODELS to your new location, move the existing models directory, then restart. For systemd-managed installations, edit /etc/systemd/system/ollama.service to add Environment=“OLLAMA_MODELS=/mnt/storage/ollama-models” under the [Service] section. After running systemctl daemon-reload and systemctl restart ollama, verify the new path with ollama list. ...

TL;DR Odysseus transforms your self-hosted infrastructure into a unified AI workspace that coordinates multiple capabilities – chat interfaces, code completion, image generation, and document analysis – through a single web interface. Unlike single-purpose tools that handle one task, Odysseus provides workspace management features designed for teams and complex projects that span multiple AI modalities. ...

TL;DR Deploying Ollama on Kubernetes transforms local AI inference into a production-grade service with horizontal scaling, persistent model storage, and service mesh integration. This guide covers container orchestration patterns specifically for LLM workloads running on self-hosted infrastructure. The core deployment uses StatefulSets rather than Deployments to maintain stable network identities and persistent volume claims for model storage. Each Ollama pod serves the REST API on port 11434 and requires GPU node affinity when using NVIDIA runtime. Configure OLLAMA_HOST=0.0.0.0:11434 to bind the service to all interfaces within the pod network, and set OLLAMA_MODELS=/models pointing to your PersistentVolume mount path. ...

TL;DR This guide walks you through deploying Stable Diffusion on your own Linux machine using ComfyUI and Automatic1111 (A1111), giving you complete control over your image generation pipeline without sending prompts or outputs to third-party services. You need an NVIDIA GPU with at least 6GB VRAM for basic operation. Cards like the RTX 3060 work well for standard 512x512 images, while RTX 4090 or A6000 cards handle larger resolutions and batch processing. AMD GPUs work through ROCm but require additional configuration. CPU generation is possible but extremely slow. ...

TL;DR This guide walks you through deploying a complete local LLM stack on Mac Mini hardware, specifically optimized for Apple Silicon’s unified memory architecture. You’ll install Ollama as your model runtime and Open WebUI as your chat interface, creating a private AI environment that keeps all data on your local network. The Mac Mini M2 Pro and M4 models excel at running 7B to 13B parameter models thanks to their high-bandwidth unified memory. Unlike traditional GPU setups, Apple Silicon shares memory between CPU and GPU cores, eliminating PCIe bottlenecks. This architecture means a Mac Mini with 32GB RAM can comfortably run llama3.1:8b or mistral:7b models while leaving headroom for the web interface and system processes. ...

TL;DR This guide shows you how to build a peer-to-peer GPU sharing network using Go-based tools that let idle machines serve AI inference requests across your local network or homelab. Instead of leaving GPUs idle on workstations overnight, you can pool them into a distributed inference cluster that routes requests to available hardware. ...

TL;DR Docker Hub rate limits and regional connectivity issues in Spain can block container pulls, disrupting self-hosted AI deployments. The primary workaround is switching to mirror registries or running Ollama natively without Docker. For immediate relief, configure Docker to use alternative registries. Edit /etc/docker/daemon.json to add registry mirrors: { "registry-mirrors": [ "https://mirror.gcr.io" ] } Restart Docker with sudo systemctl restart docker and retry your pull. This routes requests through Google’s mirror, bypassing Docker Hub entirely. ...

TL;DR # On connected machine: download everything curl -fsSL https://ollama.com/install.sh -o ollama-install.sh ollama pull llama3.1:8b tar czf ollama-models.tar.gz -C /usr/share/ollama .ollama/ # Transfer to air-gapped machine via USB # On air-gapped machine: install and restore bash ollama-install.sh # works offline if binary is bundled tar xzf ollama-models.tar.gz -C /usr/share/ollama/ sudo systemctl start ollama ollama list # verify models are available The full process involves downloading the Ollama binary, pulling models, packaging everything, transferring via approved media, and restoring on the isolated system. This guide covers each step in detail. ...

TL;DR # Run Tabby with NVIDIA GPU using Docker docker run -d --name tabby \ --gpus all \ -p 8080:8080 \ -v $HOME/.tabby:/data \ tabbyml/tabby \ serve --model StarCoder-1B --device cuda # Verify it is running curl http://localhost:8080/v1/health # Test a completion curl -X POST http://localhost:8080/v1/completions \ -H "Content-Type: application/json" \ -d '{"prompt": "def fibonacci(n):\n ", "language": "python"}' Install the Tabby plugin in your IDE, point it at http://localhost:8080, and get Copilot-style completions backed entirely by local hardware. ...

LocalAI Setup: OpenAI API-Compatible Local Inference TL;DR # Docker (quickest start) docker run -d --name localai -p 8080:8080 \ -v localai-models:/build/models \ localai/localai:latest-gpu-nvidia-cuda-12 # Install a model from the gallery curl http://localhost:8080/models/apply -d '{"id": "llama-3.1-8b-instruct"}' # Test chat completions (same as OpenAI API) curl http://localhost:8080/v1/chat/completions \ -H "Content-Type: application/json" \ -d '{"model": "llama-3.1-8b-instruct", "messages": [{"role": "user", "content": "Hello"}]}' # Generate embeddings curl http://localhost:8080/v1/embeddings \ -H "Content-Type: application/json" \ -d '{"model": "text-embedding-ada-002", "input": "The quick brown fox"}' Caution: LocalAI has no built-in authentication. Any process that can reach port 8080 can use the API. Use firewall rules, bind to localhost only, or put a reverse proxy with auth in front before exposing to a network. ...