Ollama Modelfile Guide: Custom System Prompts and Parameters

TL;DR

# Create a custom model from a Modelfile
ollama create my-coder -f ./Modelfile

# Run it
ollama run my-coder

# List custom models
ollama list | grep my-

# Remove a custom model
ollama rm my-coder

A Modelfile is a plain text file that defines a custom Ollama model. It specifies the base model, system prompt, generation parameters, and template format. Think of it as a Dockerfile for LLMs: declarative, reproducible, and version-controllable.

What Is a Modelfile

A Modelfile is the configuration format Ollama uses to create derived models. Rather than fine-tuning weights, a Modelfile layers instructions and parameters on top of an existing base model. The result is a new model entry in your local Ollama registry that behaves according to your specifications every time you run it.

This is useful when you want:

• A model that always follows a specific system prompt
• Tuned generation parameters (temperature, context window, top_p) baked in
• Specialized assistants for different tasks without remembering flags each time
• Shareable configurations that teammates can import with one command

Modelfile Directives Reference

FROM

Every Modelfile starts with FROM. This specifies the base model:

FROM llama3.2:3b

You can use any model available in your local registry (ollama list) or any model from the Ollama library. If the model is not already pulled, ollama create will pull it automatically.

You can also point to a local GGUF file:

FROM ./my-model.Q4_K_M.gguf

PARAMETER

The PARAMETER directive sets generation parameters. Each parameter goes on its own line:

PARAMETER temperature 0.3
PARAMETER top_p 0.9
PARAMETER num_ctx 8192
PARAMETER repeat_penalty 1.1
PARAMETER num_predict 2048
PARAMETER stop "<|end|>"

Key parameters and their effects:

Caution: Setting num_ctx higher than the base model supports will not extend its actual context capability. It will allocate more VRAM without benefit. Check the model’s training context length with ollama show <model> before increasing this value.

SYSTEM

The system prompt defines the model’s behavior, persona, and constraints:

SYSTEM """You are a Linux system administrator with 20 years of experience.
You provide concise, accurate answers about system administration.
Always include the exact commands needed. Specify which Linux distributions
the commands apply to when they differ. Never suggest running commands
as root unless absolutely necessary -- use sudo instead."""

Triple quotes allow multi-line system prompts. Single-line prompts use regular quotes:

SYSTEM "You are a helpful coding assistant. Always include error handling in code examples."

TEMPLATE

The template directive overrides how messages are formatted before being sent to the model. Most users should not need this – the base model’s template is correct by default. It uses Go template syntax:

TEMPLATE """{{ if .System }}<|system|>
{{ .System }}<|end|>
{{ end }}{{ if .Prompt }}<|user|>
{{ .Prompt }}<|end|>
{{ end }}<|assistant|>
{{ .Response }}<|end|>
"""

Only override this if you are using a GGUF file that Ollama does not have template metadata for, or if you are experimenting with prompt formatting.

MESSAGE

The MESSAGE directive seeds the conversation with example exchanges. This is useful for few-shot prompting:

MESSAGE user "How do I check disk usage?"
MESSAGE assistant "Run df -h for filesystem usage or du -sh /path for directory sizes."

Complete Modelfile Examples

Example 1: Sysadmin Assistant

This model answers Linux administration questions with precise commands and distribution-specific notes.

Create a file named Modelfile.sysadmin:

FROM llama3.2:3b

PARAMETER temperature 0.2
PARAMETER top_p 0.85
PARAMETER num_ctx 4096
PARAMETER repeat_penalty 1.15

SYSTEM """You are an expert Linux system administrator. Follow these rules strictly:

1. Provide exact commands, not pseudocode.
2. Specify the distribution when commands differ (Debian/Ubuntu vs RHEL/Fedora vs Arch).
3. Use sudo instead of running as root.
4. Warn about destructive operations before showing the command.
5. Include the expected output format when it helps verify the command worked.
6. For configuration file edits, show the specific lines to change, not the entire file.

Do not explain basic concepts unless asked. Assume the user is comfortable with the terminal."""

MESSAGE user "How do I find which process is using port 8080?"
MESSAGE assistant "Use ss or lsof:

ss -tlnp | grep :8080

Or:

sudo lsof -i :8080

Both work on all major distributions. The lsof command requires sudo to show process names for services running as other users."

Build and run:

ollama create sysadmin -f Modelfile.sysadmin
ollama run sysadmin "How do I resize an ext4 partition without downtime?"

Example 2: Python Code Reviewer

This model reviews code for bugs, security issues, and style problems.

Create Modelfile.code-review:

FROM qwen2.5-coder:7b

PARAMETER temperature 0.1
PARAMETER top_p 0.9
PARAMETER num_ctx 8192
PARAMETER num_predict 4096
PARAMETER repeat_penalty 1.1

SYSTEM """You are a senior Python code reviewer. When given code, analyze it for:

1. Bugs and logic errors
2. Security vulnerabilities (SQL injection, path traversal, unsafe deserialization)
3. Performance issues (unnecessary loops, missing generators, N+1 patterns)
4. Missing error handling
5. PEP 8 style violations

Format your response as:

BUGS: list any bugs found
SECURITY: list any security issues
PERFORMANCE: list any performance concerns
STYLE: list style violations
SUGGESTED FIX: show corrected code if bugs were found

If the code is correct, say so briefly. Do not add unnecessary praise."""

Build and run:

ollama create code-review -f Modelfile.code-review
ollama run code-review "$(cat my_script.py)"

Or pipe code directly:

cat my_script.py | ollama run code-review

Example 3: Technical Writer

This model generates documentation and README content in a consistent style.

Create Modelfile.tech-writer:

FROM llama3.1:8b

PARAMETER temperature 0.4
PARAMETER top_p 0.9
PARAMETER num_ctx 8192
PARAMETER num_predict 4096

SYSTEM """You are a technical documentation writer. Follow these conventions:

1. Use plain, direct language. No marketing speak or buzzwords.
2. Start with a one-sentence summary of what the tool/feature does.
3. Include prerequisites before instructions.
4. Use numbered steps for procedures, bullet points for lists.
5. Include code blocks with the correct language tag for syntax highlighting.
6. Add a Troubleshooting section when the topic involves installation or configuration.
7. Never use emojis or decorative formatting.
8. Write in second person (you/your), present tense."""

Build and run:

ollama create tech-writer -f Modelfile.tech-writer
ollama run tech-writer "Write a README for a Python CLI tool that converts CSV files to JSON"

Example 4: Constrained JSON Output

For pipeline use cases where you need structured output:

Create Modelfile.json-extractor:

FROM llama3.2:3b

PARAMETER temperature 0.0
PARAMETER top_p 0.8
PARAMETER num_ctx 4096
PARAMETER repeat_penalty 1.2

SYSTEM """You extract structured data from text and return it as valid JSON only.
Do not include any text before or after the JSON object.
Do not use markdown code fences.
If a field cannot be determined from the input, set it to null."""

MESSAGE user "Extract: John Smith, senior engineer at Acme Corp, [email protected], hired 2023"
MESSAGE assistant "{\"name\": \"John Smith\", \"title\": \"senior engineer\", \"company\": \"Acme Corp\", \"email\": \"[email protected]\", \"hire_year\": 2023}"

ollama create json-extractor -f Modelfile.json-extractor
echo "Extract: Jane Doe, CTO at StartupX, [email protected]" | ollama run json-extractor

Parameter Tuning Guidelines

Choosing parameters depends on the task. Here are baseline configurations for common use cases:

Temperature 0.0 makes the model fully deterministic (given the same seed). Use this for extraction and classification tasks where consistency matters.

num_ctx and VRAM: Each doubling of context length roughly doubles the KV cache memory usage. On an 8GB GPU running a 7B Q4 model, you can typically afford 4096-8192 context. Going to 16384 may cause OOM errors.

Inspecting and Debugging Custom Models

After creating a model, inspect it:

# Show the Modelfile used to create a model
ollama show sysadmin --modelfile

# Show model parameters
ollama show sysadmin --parameters

# Show the system prompt
ollama show sysadmin --system

# Show the template
ollama show sysadmin --template

If the model does not behave as expected, check:

1. The system prompt is actually set: ollama show <model> --system
2. Parameters took effect: ollama show <model> --parameters
3. The base model supports your context length: ollama show <base-model>

Sharing Custom Models

To share a Modelfile-based model, you have two options.

Option 1: Share the Modelfile (recommended). Check the Modelfile into version control. Recipients run ollama create themselves. This is lightweight and the recipient always gets the latest base model.

Option 2: Push to a registry. If you have an account on ollama.com:

# Tag the model with your namespace
ollama cp sysadmin yourname/sysadmin

# Push to registry
ollama push yourname/sysadmin

Others can then pull it with ollama pull yourname/sysadmin.

Security Note

System prompts are not a security boundary. A determined user can override or extract the system prompt through prompt injection. Do not rely on system prompts to enforce access controls or hide sensitive instructions. If you need to restrict model behavior in a multi-user environment, enforce constraints at the API layer (authentication, input filtering, output filtering) rather than in the Modelfile.

Version Control Best Practices

Store Modelfiles alongside your project code:

project/
  Modelfile.sysadmin
  Modelfile.code-review
  Modelfile.json-extractor
  scripts/
    setup-models.sh

Create a setup script to build all models:

#!/bin/bash
set -euo pipefail

for f in Modelfile.*; do
    name="${f#Modelfile.}"
    echo "Building model: $name"
    ollama create "$name" -f "$f"
done

This ensures every team member runs the same model configurations, and changes are tracked in git history.