Continue.dev with Ollama: Local AI Coding in VS Code

TL;DR

# Install Ollama and pull models
curl -fsSL https://ollama.com/install.sh | sh
ollama pull qwen2.5-coder:7b
ollama pull codellama:7b

# Verify Ollama is running
curl http://localhost:11434/api/tags

Install the Continue extension from the VS Code marketplace, open ~/.continue/config.json, point it at your local Ollama instance, and start coding with zero cloud dependencies.

What Is Continue.dev

Continue is an open-source AI coding assistant that integrates directly into VS Code and JetBrains IDEs. Unlike GitHub Copilot, Continue does not require a subscription or send your code to external servers. It supports multiple backends – OpenAI, Anthropic, Ollama, LM Studio, llama.cpp, and others – making it the most flexible option for developers who want local-first AI assistance.

The extension provides three core features:

• Tab completion – inline suggestions as you type, similar to Copilot
• Chat – a sidebar panel for asking questions about your codebase
• Inline editing – select code, describe what you want changed, and the model rewrites it in place

All three features can be backed by different models. A small, fast model for tab completion and a larger, more capable model for chat is the standard configuration.

Installation

VS Code

Open the Extensions panel (Ctrl+Shift+X), search for “Continue”, and install it. Alternatively:

code --install-extension Continue.continue

After installation, the Continue sidebar icon appears in the left panel. The extension creates a configuration directory at ~/.continue/.

JetBrains

Open Settings > Plugins > Marketplace, search for “Continue”, and install. Restart the IDE. Configuration is shared at the same ~/.continue/ path.

Ollama Backend

Ollama must be running and accessible at http://localhost:11434. If you installed Ollama via the standard installer, it runs as a system service automatically. Verify:

systemctl status ollama
# or
curl -s http://localhost:11434/api/tags | jq '.models[].name'

Choosing Models

Not all models are equal for coding tasks. Here is a practical breakdown:

Pull the models you want to use:

ollama pull qwen2.5-coder:7b
ollama pull deepseek-coder-v2:16b

The general recommendation: use Qwen2.5-Coder 7B for tab completion (fast enough for real-time suggestions) and DeepSeek-Coder-V2 16B for chat (where latency matters less and reasoning quality matters more). If you only have 8 GB of VRAM, stick with 7B models for everything.

Configuration

Continue stores its configuration in ~/.continue/config.json. Here is a complete working configuration for Ollama:

{
  "models": [
    {
      "title": "Qwen2.5-Coder 7B (Chat)",
      "provider": "ollama",
      "model": "qwen2.5-coder:7b",
      "apiBase": "http://localhost:11434"
    },
    {
      "title": "DeepSeek-Coder-V2 16B (Chat)",
      "provider": "ollama",
      "model": "deepseek-coder-v2:16b",
      "apiBase": "http://localhost:11434"
    }
  ],
  "tabAutocompleteModel": {
    "title": "Qwen2.5-Coder 7B (Autocomplete)",
    "provider": "ollama",
    "model": "qwen2.5-coder:7b",
    "apiBase": "http://localhost:11434"
  },
  "tabAutocompleteOptions": {
    "debounceDelay": 400,
    "maxPromptTokens": 2048,
    "disable": false
  },
  "allowAnonymousTelemetry": false
}

Key configuration notes:

• models defines the models available in the chat sidebar. You can list multiple and switch between them.
• tabAutocompleteModel is a separate field specifically for inline tab completion.
• debounceDelay controls how long Continue waits after you stop typing before requesting a completion. Lower values feel more responsive but increase GPU load. 300-500ms is the practical range.
• maxPromptTokens limits context sent to the model. Higher values give better suggestions but increase latency.
• Set allowAnonymousTelemetry to false to keep everything fully local.

Remote Ollama

If Ollama runs on a different machine (a GPU server, for example), change apiBase:

{
  "apiBase": "http://192.168.1.100:11434"
}

On the Ollama server, set the environment variable to allow remote connections:

# /etc/systemd/system/ollama.service.d/override.conf
[Service]
Environment="OLLAMA_HOST=0.0.0.0"

Then reload:

sudo systemctl daemon-reload
sudo systemctl restart ollama

Security note: Ollama has no built-in authentication. If you expose it on a network, restrict access with firewall rules or place it behind a reverse proxy with authentication. Never expose port 11434 to the public internet.

# Allow only your workstation
sudo ufw allow from 192.168.1.50 to any port 11434
sudo ufw deny 11434

Using Continue

Tab Completion

Once configured, tab completion works automatically. As you type, Continue sends the surrounding code context to the model and displays inline suggestions in gray text. Press Tab to accept, Escape to dismiss.

The quality depends heavily on the model. Qwen2.5-Coder 7B handles single-line completions, function signatures, and common patterns well. It struggles with complex multi-line logic that requires understanding distant parts of the codebase.

Chat

Open the Continue sidebar (Ctrl+L by default) and ask questions. The chat model receives your selected code (if any) plus file context. Useful patterns:

• Select a function and ask “explain this code”
• Ask “write unit tests for the selected function”
• Paste an error message and ask for debugging help

You can reference files with @filename and use @codebase to search across your project (requires indexing to be enabled in the config).

Inline Editing

Select code, press Ctrl+I, and describe the change you want. The model generates a diff that you can accept or reject. This works well for:

• Renaming variables across a selection
• Adding error handling to a function
• Converting callback-based code to async/await

Performance Tips

1. Keep the autocomplete model small. Tab completion must return results in under 500ms to feel usable. A 7B model on a decent GPU hits this target. A 16B model often does not.

2. Use FIM-capable models for tab completion. Fill-in-the-middle (FIM) models like Qwen2.5-Coder and CodeLlama are specifically trained to complete code given surrounding context. General-purpose chat models perform poorly for this task.

3. Increase debounceDelay on slow hardware. If your GPU cannot keep up, set it to 600-800ms to reduce the number of inference requests.

4. Disable tab completion when not needed. If you are writing prose or documentation, disable autocomplete temporarily to free GPU resources for chat:

{
  "tabAutocompleteOptions": {
    "disable": true
  }
}

5. Monitor GPU utilization. Tab completion fires constantly while you type. Watch for thermal throttling:

# NVIDIA
watch -n 1 nvidia-smi

# Check Ollama process specifically
ollama ps

6. Quantization tradeoffs. Default Ollama models are Q4_K_M quantized. For tab completion, this is fine – speed matters more than peak quality. For chat where you want better reasoning, pull a higher quantization if VRAM allows:

ollama pull qwen2.5-coder:7b-instruct-q8_0

Comparison with GitHub Copilot

The honest assessment: Copilot produces better suggestions because it runs much larger models on cloud infrastructure. Continue.dev with Ollama is the right choice when data privacy is non-negotiable, when you work offline, or when you want full control over the AI stack. For many developers, the quality gap is small enough that the privacy and cost benefits win.

Troubleshooting

Tab completion not appearing: Check that tabAutocompleteModel is set in config.json (not just models). Verify Ollama has the model loaded: ollama ps.

Slow responses: Run ollama ps to check if the model is loaded in memory. First request after idle may be slow due to model loading. Set OLLAMA_KEEP_ALIVE=-1 to keep models loaded indefinitely (uses VRAM permanently).

Model not found errors: Ensure the model name in config.json exactly matches what Ollama reports: ollama list.

High CPU usage with no GPU: Ollama defaults to CPU inference if no compatible GPU is detected. Check ollama --version and verify GPU drivers are installed. For NVIDIA: nvidia-smi should show the Ollama process.

Next Steps

Once Continue.dev is working, consider these improvements:

• Set up a dedicated GPU server running Ollama and point all your workstations at it
• Configure different models per programming language in Continue’s config
• Enable codebase indexing for context-aware suggestions across your entire project
• Combine with Open WebUI for a browser-based chat interface to the same Ollama instance