# Qwen3.8-Flash-Next: A Cost-Optimized Open-Weight Multimodal Model Built on a 180B-Parameter Architecture
Alibaba’s Qwen research team has introduced Qwen3.8-Flash-Next, an open-weight multimodal Mixture-of-Experts model engineered with a strong focus on cost per token. Positioned as a precursor to Qwen4, this release mirrors the architectural exploration path that Qwen3-Next paved for Qwen3.5 — pushing the boundaries of what a large model can deliver while drastically reducing per-token compute expenditure. The result is a checkpoint that carries 180 billion parameters on disk yet activates only 6 billion per token, a ratio that makes it one of the most efficient frontier-class models available today.
## Architecture at a Glance
The backbone of Qwen3.8-Flash-Next is a 125 billion parameter multimodal MoE model. Three additional components round out the checkpoint: a 51 billion parameter N-gram embedding table, a 4 billion parameter multi-token prediction module, and a MoE routing layer with 512 experts, of which 10 are routed per token along with 1 shared expert. At the expert level, each route operates at an intermediate dimension of 640.
What makes this architecture remarkable is the selective activation scheme. Despite the sheer volume of weights stored on disk, only a small fraction — roughly 3.3% — participates in computation for any given token. This sparse activation strategy is the cornerstone of the model’s cost efficiency, though it is important to understand that it reduces compute requirements without lowering the memory footprint needed to store the full weight matrix.
## Four Architectural Innovations
The release introduces four deliberate design changes that collectively define the Qwen3.8-Flash-Next experience.
### Gated DeltaNet and Qwen Sparse Attention Hybrid
The model’s 48 layers follow a repeating layout of 12 repetitions of a four-layer block: three Gated DeltaNet layers followed by one Qwen Sparse Attention layer. Gated DeltaNet is a linear attention mechanism that compresses the entire conversation history into a fixed-size recurrent state, which means memory consumption stays constant regardless of sequence length. Qwen Sparse Attention complements this by using a lightweight indexer to retrieve context at micro-block granularity rather than processing every single token individually.
This hybrid approach addresses a well-known trade-off in transformer design: linear attention is computationally cheap but struggles with precise long-range retrieval, while standard attention is accurate but expensive at scale. By dedicating one out of every four layers to sparse retrieval, Qwen3.8-Flash-Next gains the precision benefits of targeted attention without the full computational cost.
### Gated Residual Stream
The residual stream — the pathway through which information flows between layers — has been expanded into four parallel branches. Each branch is controlled by an element-wise read gate and a per-branch scalar write gate, operating at a bottleneck rank of 320. This design allows the model to maintain richer cross-layer representations while keeping training dynamics stable and inference overhead minimal.
### N-gram Embedding Table
At layer 2 of the network sits a 20 million entry bigram and trigram embedding table totaling 51 billion parameters. Because lookups in this table are deterministic, they introduce additional model capacity with virtually no per-token compute cost. The table can be offloaded to host memory with asynchronous prefetching, though this offload path currently functions only on NVIDIA hardware according to deployment documentation.
### Muon Optimizer and Training Recipe
Training Qwen3.8-Flash-Next leverages the Muon optimizer applied alongside AdamW, targeting specific weight categories within the model. The training recipe eliminates batch-size warmup phases and incorporates refitted scaling laws. The reported training cost is approximately one-ninth that of the previous Qwen3.7-Plus model, representing a significant improvement in the cost efficiency of frontier-scale training.
## Benchmark Performance
On code generation and software engineering tasks, Qwen3.8-Flash-Next scores 58.7 on DeepSWE 1.1, 62.5 on SWE-bench Pro, 81.0 on SWE-bench Multilingual, and 91.9 on LiveCodeBench v6. For agentic workflows, the model achieves 73.9 on CoWorkBench, 55.7 on JobBench, and 73.5 on Toolathlon Verified. Multimodal evaluations produce strong results as well: 84.5 on AndroidWorld, 76.6 on LVBench, 88.5 on RealWorldQA, and 95.7 on MathVision with code interpreter.
However, the model does not lead in every category. On the Humanity’s Last Exam benchmark, Claude Opus 4.6 (Max) achieves 40.0 compared to Qwen’s 35.9. Similarly, DeepSeek-V4-Flash-0731 leads the NL2Repo-Bench at 54.2 versus 48.1. These gaps highlight that frontier reasoning and coding capabilities still have room for improvement even among top-tier open models.
## Efficiency and Serving Performance
The efficiency gains extend beyond training. Qwen reports that Qwen Sparse Attention delivers kernel-level speedups of up to 7.6× for prefill operations and 4.9× for decode operations when processing sequences at approximately 1 million tokens. Third-party serving tools such as SGLang and vLLM report even more dramatic improvements at 10.2× and 6.6× respectively, though these figures should be treated as vendor-reported until independently verified.
Additionally, Qwen claims 8.6× the prefill throughput of Qwen3.7-Plus when operating at a 90% prefix-cache hit rate, and native context length of 262,144 tokens, extendable to 1 million tokens using YaRN scaling.
## Deployment Requirements
Qwen3.8-Flash-Next is deployable today but not on consumer hardware. The FP8 quantized checkpoint occupies 172.78 GiB of storage, while the BF16 version requires 335.28 GiB. Recommended deployment configurations include:
– **TP2** (tensor parallelism across 2 GPUs) as the minimum validated FP8 setup on GB300-class hardware.
– **TP4** as the recommended configuration for most production scenarios.
– **TP8** on an 8×H200 node for full throughput.
A critical note for practitioners: plain TP8 is incompatible with the checkpoint’s 128-wide quantization blocks. The sparse activation mechanism reduces compute overhead but does not reduce storage requirements, so all 180 billion parameters must reside in GPU memory or be appropriately offloaded.
## How to Run the Model
The model is compatible with several serving frameworks including vLLM, SGLang, TokenSpeed, and the `transformers serve` interface. GGUF quantizations are available for llama.cpp users. For fine-tuning, the team supports Unsloth, Swift, and LLaMA-Factory as compatible frameworks. The model also powers the “Standard” mode on QwenWork and integrates with Qwen Code for development workflows.
Thinking mode is enabled by default, with the `reasoning_effort` parameter adjustable to high, medium, or low intensity. Recommended generation settings are temperature 1.0 and top_p 0.95 for thinking mode, and temperature 0.7 with top_p 0.80 for instruct mode.
## Licensing
The model is released under the qwen-community-1.0 license. It is not Apache-2.0, so organizations planning commercial deployment should review the full license terms carefully before integrating the model into products.
—
## Frequently Asked Questions
**What is the total parameter count of Qwen3.8-Flash-Next?**
The checkpoint contains 180 billion parameters total: a 125 billion backbone, a 51 billion N-gram embedding table, and a 4 billion multi-token prediction module.
**How many parameters activate per token?**
Only approximately 6 billion parameters activate per token, representing about 3.3% of the total checkpoint.
**Can I run this model on a single workstation or consumer GPU?**
No. The FP8 checkpoint alone is 172.78 GiB, and the recommended minimum deployment is TP2 on GB300-class hardware or TP8 on an 8×H200 node.
**What is Gated DeltaNet?**
Gated DeltaNet is a linear attention layer that compresses the full conversation history into a fixed-size recurrent state, keeping memory usage constant regardless of sequence length.
**What is Qwen Sparse Attention?**
QSA is a sparse attention mechanism that uses a lightweight indexer to aggregate the input sequence into micro-blocks and retrieves only a fixed budget of 512 blocks (2,048 tokens) for precise attention computation.
**What is the N-gram Embedding table?**
A 20 million entry deterministic lookup table at layer 2 that stores bigram and trigram patterns, adding 51 billion parameters of capacity with negligible per-token compute cost.
**What is the native context window?**
262,144 tokens natively, extendable to 1,000,000 tokens with YaRN positional scaling.
**Is thinking mode on by default?**
Yes, thinking mode is the default, with reasoning effort adjustable between high, medium, and low.
**What license does the model use?**
qwen-community-1.0, which is not Apache-2.0. Commercial users should review the license terms carefully.
**How does the training cost compare to previous models?**
Training Qwen3.8-Flash-Next costs roughly one-ninth as much as Qwen3.7-Plus.
—
## Conclusion
Qwen3.8-Flash-Next represents a significant step forward in the design of cost-efficient large language models. By combining a massive 180-billion-parameter checkpoint with aggressive sparsity and architectural innovations like Gated DeltaNet, Qwen Sparse Attention, Gated Residual branching, and a large deterministic N-gram embedding table, the model achieves a compelling balance between capability and affordability. The training cost reduction to one-ninth of its predecessor and the multi-fold inference speedups through sparse attention kernels make it a compelling option for organizations that need frontier-tier performance without the infrastructure demands of a fully dense model of comparable scale. With strong benchmarks across coding, agentic tasks, and multimodal understanding, Qwen3.8-Flash-Next establishes a clear trajectory toward the Qwen4 architecture and reinforces Alibaba’s position in the open-weight frontier model landscape.
Thank you for reading
