Unlocking Claude Code’s Full Potential with Local Model Pairings

# Introduction

Running agentic coding sessions can get pricey. A single Claude Code session — involving file reading, code writing, test execution, and repeated iterations — can consume 10 to 50 times more tokens than a standard chat exchange. When you scale this up, costs accumulate quickly. Factor in rate limits that might halt a lengthy workflow midway, plus reliance on a third-party API that could alter pricing, tighten restrictions, or experience downtime at any moment, and the argument for running models locally becomes clear.

By 2026, local models have reached a solid level of quality. For the everyday tasks Claude Code manages — completing code, refactoring, debugging, and explaining codebases — a carefully selected quantized model running on your own hardware handles the overwhelming majority of real-world scenarios at no per-token expense and without any rate limits. This guide walks through three inference backends (Ollama, LM Studio, and llama.cpp), the precise environment variables and configuration files needed to connect each one to Claude Code, a handpicked table of models worth trying, and practical troubleshooting solutions for the problems you’re most likely to encounter.

# How Claude Code Connects to Any Local Model

The process is more straightforward than most tutorials suggest. Claude Code sends requests using the Anthropic Messages API format. By default, those requests are routed to Anthropic’s servers. By setting ANTHROPIC_BASE_URL, you redirect them to any server that understands the same format — which now includes Ollama, LM Studio, and llama.cpp out of the box.

According to the official Claude Code environment variables documentation, the key variables for this configuration are:

• ANTHROPIC_BASE_URL: reroutes all API calls from Anthropic’s servers to whatever URL you specify. Point this to your local inference server address.
• ANTHROPIC_API_KEY: the API key included in the request header. Local servers generally don’t enforce authentication, so this is typically set to a placeholder like “local” or “ollama.”
• ANTHROPIC_AUTH_TOKEN: an alternative authentication header. Some local servers look for this instead of the API key. Set it to the same placeholder value.

ANTHROPIC_DEFAULT_SONNET_MODEL, ANTHROPIC_DEFAULT_HAIKU_MODEL, and ANTHROPIC_DEFAULT_OPUS_MODEL: Claude Code internally requests different model tiers based on the task at hand. These three variables map each tier to the name of your local model. Without them, Claude Code sends requests for claude-sonnet-4-20250514 to your local server, which will reject the request since no such model exists on your machine.

In January 2026, Ollama introduced native support for the Anthropic Messages API — the key technical improvement that made this workflow viable without needing translation proxies. LM Studio added a native /v1/messages endpoint in version 0.4.1. llama.cpp has supported the Anthropic API directly for a longer time. All three now communicate using Claude Code’s native protocol.

A clean architecture diagram showing Claude Code, Ollama, LM Studio, and llama.cpp | Image by Author

# Backend 1: Ollama

Ollama is the ideal place to start. It abstracts away all the complexity of model management — downloading weights, quantization, GPU and CPU allocation, and serving — behind a clean command-line interface (CLI). One command to install, one command to pull a model, and a few environment variables to set up. It runs as a background service after installation, so there’s no need to manually start a server.

Prerequisites

• macOS, Linux, or Windows (WSL2 recommended on Windows)
• At least 16 GB of RAM for practical use (32 GB recommended)
• A GPU with 8+ GB VRAM for GPU-accelerated inference, or CPU-only mode with sufficient RAM
• Ollama v0.14.0 or later is required for Anthropic Messages API support

Install Ollama:

# macOS and Linux -- single command installation
curl -fsSL  | sh

# Check the version -- must be 0.14.0+ for Claude Code compatibility
ollama version
# Expected: ollama version is 0.14.x or higher

# Windows: download the installer from 
# Native Windows support has improved significantly in recent releases

Once installed, Ollama launches automatically as a background service on port 11434. You can confirm it’s running:

# Verify the Ollama server is active
curl 

# Expected response:
# Ollama is running

Download a coding model:

# GLM-4.7-Flash -- recommended starting point
# Strong tool calling, 128K context window, fits in 8 GB VRAM
# Apache 2.0 license
ollama pull glm-4.7-flash:latest

# Qwen3-Coder -- excellent code generation and instruction following
# Requires 20+ GB VRAM for the full model
ollama pull qwen3-coder

# Devstral-Small -- purpose-built for agentic coding workflows
# Community-tested for Claude Code compatibility
# 24B parameters, requires 16+ GB VRAM
ollama pull devstral-small-2:24b

# Confirm the model is downloaded and ready to use
ollama list
# Displays all pulled models with their sizes and last modified dates

// Configuring Claude Code to Use Ollama

Option 1: Shell export (applies to the current terminal session only)

# Point Claude Code to your local Ollama server
export ANTHROPIC_BASE_URL="

# Local servers don't require real authentication
# Set these to any non-empty string -- Ollama ignores the actual value
export ANTHROPIC_API_KEY="ollama"
export ANTHROPIC_AUTH_TOKEN="ollama"

# Map Claude Code's model tier requests to your local model name
# Claude Code internally requests sonnet/haiku/opus -- these variables
# translate those tier names to whichever model you've pulled locally
export ANTHROPIC_DEFAULT_SONNET_MODEL="glm-4.7-flash:latest"
export ANTHROPIC_DEFAULT_HAIKU_MODEL="glm-4.7-flash:latest"
export ANTHROPIC_DEFAULT_OPUS_MODEL="glm-4.7-flash:latest"

# Launch Claude Code -- it will now route through Ollama instead of the Anthropic API
claude

Option 2: ~/.claude/settings.json (permanent, applies to all sessions)

This method persists across terminal restarts and takes effect every time you launch Claude Code. Claude Code reads environment variables from

{
  "env": {
    "ANTHROPIC_BASE_URL": "
    "ANTHROPIC_API_KEY": "ollama",
    "ANTHROPIC_AUTH_TOKEN": "ollama",
    "ANTHROPIC_DEFAULT_SONNET_MODEL": "glm-4.7-flash:latest",
    "ANTHROPIC_DEFAULT_HAIKU_MODEL": "glm-4.7-flash:latest",
    "ANTHROPIC_DEFAULT_OPUS_MODEL": "glm-4.7-flash:latest"
  }
}

# .env in your project root -- automatically picked up by Claude Code
ANTHROPIC_BASE_URL=
ANTHROPIC_API_KEY=ollama
ANTHROPIC_AUTH_TOKEN=ollama
ANTHROPIC_DEFAULT_SONNET_MODEL=qwen3-coder
ANTHROPIC_DEFAULT_HAIKU_MODEL=qwen3-coder
ANTHROPIC_DEFAULT_OPUS_MODEL=qwen3-coder

# Launch Claude Code with a simple test
claude

# Inside Claude Code, enter a basic prompt:
# > What model are you running?
# The local model should respond without hitting any Anthropic API endpoints.

# To make sure no external requests are sent, enable verbose logging:
claude --verbose

# Watch for lines indicating requests going to localhost:11434
# instead of api.anthropic.com

curl -fsSL https://ollama.com/install.sh | sh          # 1. Install Ollama
ollama pull glm-4.7-flash:latest                       # 2. Pull the model (~4 GB)
export ANTHROPIC_BASE_URL="http://localhost:11434"     # 3. Redirect Claude Code
export ANTHROPIC_API_KEY="ollama"                      # 4. Set placeholder auth
export ANTHROPIC_AUTH_TOKEN="ollama"
export ANTHROPIC_DEFAULT_SONNET_MODEL="glm-4.7-flash:latest"
export ANTHROPIC_DEFAULT_HAIKU_MODEL="glm-4.7-flash:latest"
export ANTHROPIC_DEFAULT_OPUS_MODEL="glm-4.7-flash:latest"
claude                                                  # 5. Launch

# For a server or VM without a GUI — use the CLI installer
curl -fsSL https://install.lmstudio.ai/linux/installer.sh | bash

# Or grab the desktop app from https://lmstudio.ai for GUI-based use

# Point the base URL to LM Studio's local server
export ANTHROPIC_BASE_URL="http://localhost:1234"
export ANTHROPIC_API_KEY="lm-studio"
export ANTHROPIC_AUTH_TOKEN="lm-studio"

# Use the exact model name as shown by LM Studio
# Copy it carefully -- including any version suffix or quantization tag
export ANTHROPIC_DEFAULT_SONNET_MODEL="qwen2.5-coder-32b-instruct"
export ANTHROPIC_DEFAULT_HAIKU_MODEL="qwen2.5-coder-32b-instruct"
export ANTHROPIC_DEFAULT_OPUS_MODEL="qwen2.5-coder-32b-instruct"

{
  "env": {
    "ANTHROPIC_BASE_URL": "http://localhost:1234",
    "ANTHROPIC_API_KEY": "lm-studio",
    "ANTHROPIC_AUTH_TOKEN": "lm-studio",
    "ANTHROPIC_DEFAULT_SONNET_MODEL": "qwen2.5-coder-32b-instruct",
    "ANTHROPIC_DEFAULT_HAIKU_MODEL": "qwen2.5-coder-32b-instruct",
    "ANTHROPIC_DEFAULT_OPUS_MODEL": "qwen2.5-coder-32b-instruct"
  }
}

# 1. Start the LM Studio server from the GUI (Local Server tab > Start Server)
# 2. Set environment variables
export ANTHROPIC_BASE_URL="http://localhost:1234"
export ANTHROPIC_API_KEY="lm-studio"
export ANTHROPIC_AUTH_TOKEN="lm-studio"
export ANTHROPIC_DEFAULT_SONNET_MODEL="your-model-name-here"
export ANTHROPIC_DEFAULT_HAIKU_MODEL="your-model-name-here"
export ANTHROPIC_DEFAULT_OPUS_MODEL="your-model-name-here"
# 3. Launch
claude

# macOS -- Homebrew is the easiest path
brew install llama.cpp

# Linux with CUDA -- building from source gives the best GPU performance
git clone https://github.com/ggerganov/llama.cpp
cd llama.cpp
cmake -B build -DGGML_CUDA=ON          # Turn on CUDA support
cmake --build build --config Release   # Compile
# Binaries end up in ./build/bin/

# Linux CPU-only build
cmake -B build
cmake --build build --config Release