Meet North Mini Code: Cohere’s Compact 3B Active Modes Power a Stellar Agentic Coding Model

This week, the Cohere AI team launched its first coding model designed for developers, called ‘North Mini Code’. This open-weight model is built specifically for software engineers. It uses a mixture-of-experts (MoE) architecture with 30 billion total parameters, though only 3 billion are activated for each token.

The launch centers on the concept of “sovereign” AI. The core idea is straightforward: operate powerful models according to your own requirements. Compact, efficient coding models enable teams to host them independently without needing massive GPU clusters. North Mini Code is designed to fill this exact need.

North Mini Code

North Mini Code is a model with 30 billion total parameters and 3 billion active ones (A3B). Cohere tailored it for three primary functions: generating code, agentic software engineering, and terminal-based tasks. It accepts text input and produces text output only, with no support for images or video.

It supports a context window of 256,000 tokens and can generate up to 64,000 tokens in a single output. Cohere specifies a minimum hardware requirement of one H100 GPU running at FP8 precision. The model weights are available under the Apache 2.0 license on Hugging Face. You can also access it via the Cohere API, Model Vault, and OpenRouter.

The Architecture

North Mini Code is a decoder-only Transformer that incorporates sparse MoE layers. Its attention mechanism alternates between two types in a 3:1 ratio. Sliding-window attention employs RoPE for positional encoding, while global attention uses no positional embeddings whatsoever. The feed-forward block contains 128 experts, with 8 activated per token. Each expert is a feed-forward network using SwiGLU activation.

The routing mechanism applies a sigmoid function before performing top-k selection. A single dense layer precedes the sparse layers. This combination keeps the active computation low while expanding the overall model capacity. Cohere released the weights in BF16 format.

Post-training was conducted in two stages. The first involved two-stage cascaded supervised fine-tuning (SFT), followed by reinforcement learning with verifiable rewards (RLVR). The post-training emphasized agentic coding capabilities. The model also features support for interleaved thinking and native tool use.

Benchmarks

Cohere reports a score of 33.4 on the Artificial Analysis Coding Index, describing this as a competitive standing among models of similar size. The company tested on SWE-Bench Verified, SWE-Bench Pro, and Terminal-Bench v2, along with Terminal-Bench Hard, SciCode, and LiveCodeBench v6.

The evaluation methodology was precise. SWE-Bench used the SWE-agent harness v1.1.0. Terminal-Bench v2 used a basic ReAct harness with a single terminal tool. Terminal-Bench Hard used the Terminus-2 harness. Each benchmark was run with three different seeds and the results were averaged. Sampling was configured with temperature 1.0 and top_p 0.95.

The Speed

In Cohere’s internal benchmarks, North Mini Code achieved up to 2.8 times greater output throughput under identical concurrency and hardware conditions. It also demonstrated a 30% improvement in inter-token latency. Time-to-first-token was comparable between the two models, with Devstral Small 2 maintaining a slight advantage in TTFT.

Use Cases With Examples

Cohere designed North Mini Code for agentic workflows.

Three key patterns emerge from its own positioning:

• Sub-agent orchestration: A primary agent assigns subtasks to supporting agents. For instance, one agent writes unit tests while another resolves failing code.
• Systems architecture mapping: The model analyzes a repository and outlines its structure. For example, mapping how services interact with each other before undertaking a major refactor.
• Code reviews: The model examines a diff for potential issues. For example, identifying an unguarded null dereference before merging code.

Terminal tasks are also well-suited to the model. For example: listing files, executing a build, and then parsing the output for errors.

Getting Started

The quickest way to begin is with Hugging Face Transformers. Install Transformers from source for this particular model. The recommended sampling settings are temperature 1.0 and top_p 0.95.

# Install Transformers from source (required for this model):
# pip install "git+
from transformers import AutoTokenizer, AutoModelForCausalLM

model_id = "CohereLabs/North-Mini-Code-1.0"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(model_id, device_map="auto")

prompt = "Write a python program to check if a string is a palindrome or not."
messages = [{"role": "user", "content": prompt}]

# return_dict=True yields a dict (input_ids + attention_mask) so **inputs unpacks cleanly
inputs = tokenizer.apply_chat_template(
    messages,
    tokenize=True,
    add_generation_prompt=True,
    return_dict=True,
    return_tensors="pt",
).to(model.device)

gen_tokens = model.generate(
    **inputs,
    max_new_tokens=1024,
    do_sample=True,
    temperature=1.0,
    top_p=0.95,
)

# Decode only the newly generated tokens, not the prompt
output = tokenizer.decode(gen_tokens[0][inputs["input_ids"].shape[-1]:])
print(output)

For serving the model, vLLM is compatible. You will need the latest vLLM main branch along with Cohere’s melody library, which is essential for accurate response parsing.

uv pip install "git+
uv pip install "cohere_melody>=0.9.0"

vllm serve CohereLabs/North-Mini-Code-1.0 
  -tp 2 
  --max-model-len 320000 
  --tool-call-parser cohere_command4 
  --reasoning-parser cohere_command4 
  --enable-auto-tool-choice

Quantized versions are available for Ollama, LM Studio, and llama.cpp. You can also test the model before downloading it. Cohere provides free access through OpenCode and a hosted Hugging Face Space.

Key Takeaways

Overview
How It Works
Performance
Get Started

# Install Transformers from source, then:
from transformers import AutoTokenizer, AutoModelForCausalLM

mid = "CohereLabs/North-Mini-Code-1.0"
tok = AutoTokenizer.from_pretrained(mid)
model = AutoModelForCausalLM.from_pretrained(mid, device_map="auto")

msgs = [{"role": "user", "content": "Write a Python palindrome checker."}]
inputs = tok.apply_chat_template(
    msgs, add_generation_prompt=True,
    return_dict=True, return_tensors="pt",
).to(model.device)

out = model.generate(**inputs, max_new_tokens=1024,
                     do_sample=True, temperature=1.0, top_p=0.95)
print(tok.decode(out[0][inputs["input_ids"].shape[-1]:]))