TL;DR
# On connected machine: download everything
curl -fsSL https://ollama.com/install.sh -o ollama-install.sh
ollama pull llama3.1:8b
tar czf ollama-models.tar.gz -C /usr/share/ollama .ollama/
# Transfer to air-gapped machine via USB
# On air-gapped machine: install and restore
bash ollama-install.sh # works offline if binary is bundled
tar xzf ollama-models.tar.gz -C /usr/share/ollama/
sudo systemctl start ollama
ollama list # verify models are available
The full process involves downloading the Ollama binary, pulling models, packaging everything, transferring via approved media, and restoring on the isolated system. This guide covers each step in detail.
Why Air-Gapped AI
Air-gapped networks have no connection to the public internet. They exist in environments where data must never leave a controlled perimeter:
- Government and military classified systems (SIPR, JWICS, or national equivalents)
- Healthcare systems handling PHI under HIPAA where network segmentation is required
- Financial trading systems with strict regulatory isolation
- Industrial control systems (SCADA/ICS) in utilities and manufacturing
- Legal and compliance environments processing privileged or export-controlled data
Running LLMs locally in these environments means you get AI capabilities without any data leaving the network. No API calls, no telemetry, no external dependencies after initial deployment.
Prerequisites
You need two machines:
| Role | Requirements |
|---|---|
| Machine A (connected) | Internet access, Ollama installed, enough disk for models |
| Machine B (air-gapped) | Same OS/architecture as Machine A, no internet |
And an approved transfer medium: USB drive, optical media, or a one-way data diode depending on your security policy.
Security note: All transfers into an air-gapped environment should follow your organization’s data transfer procedures. Most classified environments require media scanning, write-protection, and chain-of-custody documentation. This guide covers the technical steps only.
Step 1: Download the Ollama Binary
On Machine A (connected):
# Download the install script for later use
curl -fsSL https://ollama.com/install.sh -o ollama-install.sh
# Download the binary directly (Linux x86_64)
curl -fsSL https://ollama.com/download/ollama-linux-amd64 -o ollama
chmod +x ollama
# For ARM64 systems
curl -fsSL https://ollama.com/download/ollama-linux-arm64 -o ollama
chmod +x ollama
# Verify the download
sha256sum ollama > ollama.sha256
cat ollama.sha256
Record the SHA256 hash. You will verify it on the air-gapped machine after transfer.
Step 2: Pull Models on the Connected Machine
# Pull the models you need
ollama pull llama3.1:8b
ollama pull llama3.2:3b
ollama pull nomic-embed-text # embedding model for RAG
# Verify they loaded correctly
ollama list
Output:
NAME ID SIZE MODIFIED
llama3.1:8b a]2c6b7e3 4.7 GB 2 minutes ago
llama3.2:3b b1f2d3e4f5 2.0 GB 1 minute ago
nomic-embed-text:latest c6d7e8f9a0 274 MB 30 seconds ago
Step 3: Understand the Model Storage Layout
Ollama stores models under a single directory. Knowing the layout helps you verify the transfer is complete.
# Default model storage location
ls -la /usr/share/ollama/.ollama/models/
# Structure:
# /usr/share/ollama/.ollama/models/
# blobs/ # Actual model weights (large files, SHA256-named)
# manifests/ # Model metadata and layer references
# registry.ollama.com/
# library/
# llama3.1/
# 8b # Manifest for llama3.1:8b
The blobs/ directory contains the actual model weights as content-addressed files. The manifests/ directory maps model names to blob references. Both directories are required for a working offline install.
# Check total size
du -sh /usr/share/ollama/.ollama/models/
# Example output: 7.0G
# List all blobs
ls -lhS /usr/share/ollama/.ollama/models/blobs/ | head -10
Step 4: Package Everything for Transfer
# Create a transfer directory
mkdir -p ~/ollama-transfer
# Copy the binary
cp ollama ~/ollama-transfer/
cp ollama.sha256 ~/ollama-transfer/
# Package all models
tar czf ~/ollama-transfer/ollama-models.tar.gz \
-C /usr/share/ollama .ollama/
# Generate checksums for everything
cd ~/ollama-transfer
sha256sum * > CHECKSUMS.sha256
cat CHECKSUMS.sha256
If your transfer medium has a size limit (e.g., 4 GB FAT32 USB), split the archive:
# Split into 3.5 GB chunks
split -b 3500m ollama-models.tar.gz ollama-models.tar.gz.part-
sha256sum ollama-models.tar.gz.part-* >> CHECKSUMS.sha256
Copy the entire ollama-transfer/ directory to your USB drive or approved media:
# Mount USB and copy
sudo mount /dev/sdb1 /mnt/usb
cp -rv ~/ollama-transfer/* /mnt/usb/
sync
sudo umount /mnt/usb
Step 5: Install on the Air-Gapped Machine
On Machine B (air-gapped), mount the transfer media and verify integrity first.
sudo mount /dev/sdb1 /mnt/usb
cd /mnt/usb
# Verify checksums
sha256sum -c CHECKSUMS.sha256
# Every line should say "OK"
Install the binary:
# Copy binary to system path
sudo cp ollama /usr/local/bin/ollama
sudo chmod +x /usr/local/bin/ollama
# Verify
ollama --version
Restore the models:
# If you split the archive, reassemble first
cat ollama-models.tar.gz.part-* > ollama-models.tar.gz
# Create the ollama user (matches the install script behavior)
sudo useradd -r -s /bin/false -m -d /usr/share/ollama ollama
sudo mkdir -p /usr/share/ollama/.ollama/models
sudo tar xzf ollama-models.tar.gz -C /usr/share/ollama/
sudo chown -R ollama:ollama /usr/share/ollama/.ollama
Create the systemd service file:
sudo tee /etc/systemd/system/ollama.service > /dev/null <<'EOF'
[Unit]
Description=Ollama Service
After=network-online.target
[Service]
ExecStart=/usr/local/bin/ollama serve
User=ollama
Group=ollama
Restart=always
RestartSec=3
Environment="HOME=/usr/share/ollama"
Environment="OLLAMA_HOST=127.0.0.1:11434"
[Install]
WantedBy=default.target
EOF
sudo systemctl daemon-reload
sudo systemctl enable ollama
sudo systemctl start ollama
Verify:
ollama list
ollama run llama3.1:8b "What is 2+2?"
Step 6: Verify Model Integrity
After transfer, verify that model files are not corrupted. Ollama uses SHA256 content addressing, so the blob filenames themselves are the checksums.
# Verify each blob matches its filename
cd /usr/share/ollama/.ollama/models/blobs/
for f in sha256-*; do
expected="${f#sha256-}"
actual=$(sha256sum "$f" | awk '{print $1}')
if [ "$expected" != "$actual" ]; then
echo "CORRUPT: $f (expected $expected, got $actual)"
else
echo "OK: $f"
fi
done
If any file reports CORRUPT, re-transfer that specific blob from the connected machine.
Step 7: Create Custom Models with Modelfiles
In an air-gapped environment, you cannot pull models from the registry. Use Modelfiles to create custom configurations from models already on disk:
# Create a Modelfile for a customized assistant
cat > /tmp/SecureAssistant.Modelfile <<'EOF'
FROM llama3.1:8b
PARAMETER temperature 0.3
PARAMETER num_ctx 4096
SYSTEM """You are a technical assistant deployed in a secure environment.
Do not reference external URLs, services, or suggest internet-based solutions.
All recommendations must work in an isolated offline environment."""
EOF
ollama create secure-assistant -f /tmp/SecureAssistant.Modelfile
ollama run secure-assistant "How do I check disk SMART status?"
This works because the FROM directive references a model already in the local store.
Step 8: Deploy Open WebUI in Air-Gapped Mode
Open WebUI provides a browser-based interface for Ollama. In an air-gapped setup, you need to pre-pull the Docker image on the connected machine.
On Machine A (connected):
# Pull the Open WebUI image
docker pull ghcr.io/open-webui/open-webui:main
# Save it to a tar file
docker save ghcr.io/open-webui/open-webui:main -o open-webui.tar
# Generate checksum
sha256sum open-webui.tar > open-webui.tar.sha256
Transfer open-webui.tar and its checksum to the air-gapped machine.
On Machine B (air-gapped):
# Verify checksum
sha256sum -c open-webui.tar.sha256
# Load the image
docker load -i open-webui.tar
# Verify
docker images | grep open-webui
# Run Open WebUI
docker run -d \
--name open-webui \
--network host \
-v open-webui-data:/app/backend/data \
-e OLLAMA_BASE_URL=http://127.0.0.1:11434 \
-e WEBUI_AUTH=true \
-e ENABLE_SIGNUP=false \
-e ENABLE_COMMUNITY_SHARING=false \
-e SAFE_MODE=true \
ghcr.io/open-webui/open-webui:main
Key environment variables for air-gapped operation:
| Variable | Value | Purpose |
|---|---|---|
ENABLE_COMMUNITY_SHARING | false | Disables attempts to reach external sharing endpoints |
SAFE_MODE | true | Restricts features that require internet |
ENABLE_SIGNUP | false | Prevents unauthorized account creation |
WEBUI_AUTH | true | Requires authentication |
Access the UI at http://localhost:8080. Create an admin account on first launch.
Step 9: Docker-Based Ollama Deployment
If your air-gapped environment uses containers exclusively, export the Ollama Docker image instead of the standalone binary.
On Machine A (connected):
# Pull the Ollama Docker image
docker pull ollama/ollama:latest
# Pull models inside the container
docker run -d --name ollama-temp ollama/ollama
docker exec ollama-temp ollama pull llama3.1:8b
docker exec ollama-temp ollama pull llama3.2:3b
# Commit the container with models baked in
docker commit ollama-temp ollama-airgap:latest
docker stop ollama-temp && docker rm ollama-temp
# Export the image
docker save ollama-airgap:latest -o ollama-airgap.tar
sha256sum ollama-airgap.tar > ollama-airgap.tar.sha256
On Machine B (air-gapped):
sha256sum -c ollama-airgap.tar.sha256
docker load -i ollama-airgap.tar
docker run -d --name ollama --gpus all -p 11434:11434 ollama-airgap:latest
Caution: Baking models into the Docker image creates large images (5-50+ GB). For environments with many models, use a volume mount approach instead and transfer the model directory separately.
Step 10: Maintaining and Updating Models
Updating models in an air-gapped environment requires repeating the transfer process. Establish a routine:
Version tracking:
# On the air-gapped machine, record current state
ollama list > /var/log/ollama-model-inventory-$(date +%Y%m%d).txt
Update process:
- On Machine A, pull updated models:
ollama pull llama3.1:8b - Package only the changed blobs (differential transfer):
# List current blobs on Machine A
ls /usr/share/ollama/.ollama/models/blobs/ | sort > /tmp/new-blobs.txt
# Compare with the previous inventory from the air-gapped machine
comm -13 /tmp/old-blobs.txt /tmp/new-blobs.txt > /tmp/delta-blobs.txt
# Package only new blobs
mkdir -p ~/ollama-update/blobs
while read blob; do
cp "/usr/share/ollama/.ollama/models/blobs/$blob" ~/ollama-update/blobs/
done < /tmp/delta-blobs.txt
# Also copy updated manifests
cp -r /usr/share/ollama/.ollama/models/manifests ~/ollama-update/
- Transfer and apply on Machine B:
sudo cp ~/ollama-update/blobs/* /usr/share/ollama/.ollama/models/blobs/
sudo cp -r ~/ollama-update/manifests/* /usr/share/ollama/.ollama/models/manifests/
sudo chown -R ollama:ollama /usr/share/ollama/.ollama/
sudo systemctl restart ollama
ollama list
Cleanup:
# Remove models you no longer need
ollama rm old-model:tag
# Check disk usage
du -sh /usr/share/ollama/.ollama/models/
Troubleshooting Air-Gapped Issues
| Problem | Cause | Fix |
|---|---|---|
ollama list shows no models | Wrong extraction path | Verify models are under /usr/share/ollama/.ollama/models/ |
| Model runs but gives garbage | Corrupted transfer | Run the SHA256 blob verification script above |
ollama serve fails at startup | Missing library dependencies | Run ldd /usr/local/bin/ollama and install missing libs |
| GPU not detected | Driver not installed offline | Transfer and install GPU drivers before Ollama |
| Open WebUI shows connection error | Ollama not running or wrong URL | Verify OLLAMA_BASE_URL and check curl http://localhost:11434 |
Docker --gpus flag fails | NVIDIA Container Toolkit missing | Install toolkit offline (download .deb packages on Machine A) |
For persistent issues, check the Ollama logs:
journalctl -u ollama -n 100 --no-pager
All error messages will reference local paths and resources. There are no external dependencies once the installation is complete. That is the entire point.
