Getting Started With Ollama

TL;DR The ollama serve command launches the Ollama daemon that exposes a REST API on port 11434 for running local LLM inference. Unlike the simpler ollama run command for interactive chat, serve mode is designed for persistent server deployments where multiple applications need programmatic access to your models. After installing Ollama with curl -fsSL https://ollama.com/install.sh | sh, the service typically starts automatically via systemd on Linux. You can verify it’s running with systemctl status ollama or by checking if port 11434 responds to API requests. The daemon loads models on-demand when applications request them through the HTTP API. ...

TL;DR Tiny LLMs (1-3 billion parameters) let you run capable AI models on modest hardware without cloud dependencies. Unlike larger models requiring expensive GPUs, tiny models run smoothly on consumer laptops, Raspberry Pi 5 devices, and older workstations with 8GB RAM. This guide shows you how to deploy them locally using Ollama. ...

TL;DR # On connected machine: download everything curl -fsSL https://ollama.com/install.sh -o ollama-install.sh ollama pull llama3.1:8b tar czf ollama-models.tar.gz -C /usr/share/ollama .ollama/ # Transfer to air-gapped machine via USB # On air-gapped machine: install and restore bash ollama-install.sh # works offline if binary is bundled tar xzf ollama-models.tar.gz -C /usr/share/ollama/ sudo systemctl start ollama ollama list # verify models are available The full process involves downloading the Ollama binary, pulling models, packaging everything, transferring via approved media, and restoring on the isolated system. This guide covers each step in detail. ...

TL;DR Quick diagnostic commands for the most common Ollama problems: # Check if Ollama is running systemctl status ollama curl http://localhost:11434/api/version # Check GPU detection ollama ps nvidia-smi # NVIDIA rocm-smi # AMD # Check disk space for model downloads df -h ~/.ollama # Check memory available free -h # View Ollama logs journalctl -u ollama -n 50 --no-pager # Force CPU-only mode if GPU is broken OLLAMA_NUM_GPU=0 ollama serve If you are running into an issue not covered here, the Ollama logs are almost always the fastest path to a diagnosis. Start there. ...

TL;DR # Install Ollama on Raspberry Pi (ARM64) curl -fsSL https://ollama.com/install.sh | sh # Pull a model that actually works on Pi ollama pull qwen2.5:0.5b ollama pull phi3:mini # Test it ollama run qwen2.5:0.5b "Write a Python function to read a CSV file" # Check memory usage ollama ps free -h Raspberry Pi 5 with 8 GB RAM can run models up to 3B parameters at usable speeds. Stick to 0.5B-1.5B models for interactive use. Anything above 7B is not practical. ...

Ollama Behind Nginx Reverse Proxy: SSL and Multi-User Setup TL;DR # Install Nginx and Certbot sudo apt install nginx certbot python3-certbot-nginx # Get SSL certificate sudo certbot --nginx -d ollama.example.com # Test the proxy curl -s https://ollama.example.com/api/tags \ -u admin:password | jq . By default, Ollama listens on localhost:11434 with no authentication, no encryption, and no rate limiting. This is fine for single-user local development but inadequate for team use or any network-exposed deployment. Nginx solves all three problems as a reverse proxy layer in front of Ollama. ...

Ollama Model Management: Pull, Create, Copy, and Remove TL;DR # Pull a model ollama pull llama3.2:3b # List all local models ollama list # Show model details (parameters, template, license) ollama show llama3.2:3b # Copy/rename a model ollama cp llama3.2:3b my-llama # Remove a model ollama rm llama3.2:3b # Check disk usage of model storage du -sh /usr/share/ollama/.ollama/models/ Ollama stores models as layered blobs, similar to Docker images. Understanding how models are stored, tagged, and shared lets you manage disk space effectively and avoid downloading duplicate data. ...

Ollama Modelfile Guide: Custom System Prompts and Parameters TL;DR # Create a custom model from a Modelfile ollama create my-coder -f ./Modelfile # Run it ollama run my-coder # List custom models ollama list | grep my- # Remove a custom model ollama rm my-coder A Modelfile is a plain text file that defines a custom Ollama model. It specifies the base model, system prompt, generation parameters, and template format. Think of it as a Dockerfile for LLMs: declarative, reproducible, and version-controllable. ...

TL;DR Running Ollama on Windows requires different considerations than Linux deployments. You have two main paths: native Windows installation or WSL2. Native Windows offers simpler GPU access through NVIDIA CUDA or AMD ROCm drivers, while WSL2 provides a Linux-like environment but adds complexity for GPU passthrough. The native Windows installer downloads from ollama.com and runs as a system service. After installation, Ollama serves models on port 11434 and appears in your system tray. Windows Defender Firewall blocks external connections by default – you must create an inbound rule for port 11434 if accessing from other machines on your network. ...

Running llama.cpp Server for Local AI Inference TL;DR llama.cpp server mode transforms the C/C++ inference engine into a production-ready HTTP API server that handles concurrent requests with OpenAI-compatible endpoints. Instead of running single inference sessions, llama-server lets you deploy local LLMs as persistent services that multiple applications can query simultaneously. ...