# LFM2.5-VL-3B-DSpark: Liquid AI’s Experimental Speculative Decoder Cuts VLM Latency Dramatically
Liquid AI has unveiled LFM2.5-VL-3B-DSpark, a lightweight speculative-decoding draft model designed to sit alongside its LFM2.5-VL-3B vision-language model. The drafter introduces roughly 280 million additional parameters and works by proposing multiple tokens ahead of the main model, which then verifies the entire block in a single forward pass. The approach yields impressive latency reductions without altering the model’s final outputs.
The team reports decoding speedups of up to 3.13x on Apple silicon hardware and up to 2.66x on a single NVIDIA H100 GPU, making this one of the most notable efficiency gains demonstrated for vision-language models at this scale.
## What Is Speculative Decoding and Why Does It Matter for VLMs?
Standard language and vision-language models generate tokens one at a time, processing each through the full model stack per forward pass. Speculative decoding introduces a smaller, faster “drafter” network that proposes a sequence of tokens in advance. The larger target model then evaluates the entire proposed block in one pass, accepting the tokens it would have generated natively and discarding the rest.
For vision-language models, this technique has historically been harder to implement because the input pipeline involves image encoding, vision-token generation, and cross-modal attention. Liquid AI’s key insight is that by the time data reaches the hidden layers of the model, both image patches and text tokens exist as the same kind of tensor representation. This means the exact same speculative decoding algorithm used for text-only models transfers directly to vision-language workloads without modification.
The LFM2.5-VL-3B-DSpark drafter follows the design principles outlined in Liquid AI’s earlier DSpark research for text models. It reads hidden states from several layers of the target model and uses them to predict the next several tokens in the sequence.
## Drafter Architecture: Small, Focused, Efficient
The DSpark drafter is an attention-only model that keeps the design deliberately simple. Through systematic ablation studies, the team settled on four decoder layers with a block size of nine tokens as the sweet spot for performance and accuracy.
A notable architectural choice is that the drafter shares both its embedding layer and its language model head with the target model. This means the drafter carries no independent tokenizer vocabulary or output projection, keeping its footprint minimal.
| Component | Parameters |
|—|—|
| Decoder stack (4 layers) | 193.0M |
| Hidden-state projection | 21.0M |
| Markov head | 65.5M |
| Norms + confidence head | 6.4k |
| **Total Drafter Parameters** | **279.5M** |
Because the embedding and LM head are shared, the drafter adds approximately 8.9% to the total deployed parameter count of the LFM2.5-VL-3B model. The entire training process was completed over 10 epochs using supervised fine-tuning on common vision-language tasks, with all experiments and ablation studies conducted on AMD hardware.
## Measured Performance Across Hardware Platforms
Benchmarks were conducted using the MMSpec evaluation suite across six task categories: General Visual Question Answering, Text VQA, Image Captioning, Chart VQA, Complex Reasoning, and Multi-turn Conversation. Every run used a batch size of one, greedy decoding, temperature set to zero, and 16-bit precision for both the vision encoder and the model backbone.
### Benchmark Summary
| Hardware Stack | Decode Speedup | End-to-End Speedup | Tokens Accepted Per Pass |
|—|—|—|—|
| MLX-VLM, Apple M5 Max (block 8) | 2.30x – 3.13x | 1.56x – 2.62x | 3.24 – 4.34 |
| llama.cpp, Apple M3 Ultra (block 8) | 1.57x – 2.14x | 1.30x – 1.77x | 3.31 – 4.50 |
| SGLang, 1x H100 80GB (block 9) | 2.04x – 2.66x | 1.64x – 2.27x | 3.46 – 4.57 |
The top decode speedup and top end-to-end speedup often come from different tasks on the same hardware. For example, the M5 Max achieves its peak 3.13x decode speedup on the COCO image captioning task, while its best end-to-end gain of 2.62x comes from the more demanding MMMU-Pro reasoning task.
Acceptance rates across the two Apple silicon platforms were remarkably consistent, leading the team to conclude that token acceptance depends more on the drafter design and the nature of the workload than on the specific runtime or inference framework being used.
The DSpark approach also maintained its throughput advantage at elevated concurrency levels on the H100, though the gap between accelerated and unaccelerated decoding did narrow as more requests were processed simultaneously.
## Output Quality and Sampling Behavior
Under greedy decoding, the target model verifies every token proposed by the drafter, so the output is byte-for-byte identical to the base model running without any speculation. This is a significant advantage for applications where determinism matters.
When using temperature-based sampling, speculative decoding preserves the target model’s output distribution, a property formally proven in prior research. However, higher temperatures do reduce the technique’s efficiency. When probability mass is spread across more candidate tokens, the drafter and target model disagree more frequently, which lowers the acceptance rate and reduces throughput gains.
## Why End-to-End Gains Are More Modest Than Decode Gains
One of the most important nuances is that speculative decoding only accelerates the token generation phase. Image encoding and the prompt prefill phase execute at exactly the same speed with or without the drafter. On edge devices with limited compute, prefill can represent a substantial fraction of total latency.
This dynamic is a straightforward application of Amdahl’s law: the overall speedup of a system is constrained by the fraction of work that remains unaccelerated. For vision-language models, image encoding and the processing of hundreds of visual tokens alongside the text prompt mean that a significant portion of inference time is untouched by speculative decoding. This explains why the M5 Max delivers 2.69x faster decoding on TextVQA but only 1.56x end-to-end improvement.
## Deployment and Availability
The model weights are available now on Hugging Face in both Safetensors and GGUF formats. Day-one integration support has been shipped in three major inference frameworks: SGLang (version 0.5.19 or newer), MLX-VLM (version 0.7.2 or newer on Apple silicon), and llama.cpp.
To use DSpark with SGLang, launch the base LFM2.5-VL-3B model with the `–speculative-algorithm DSPARK` flag and point `–speculative-draft-model-path` to the drafter weights. On Apple silicon with MLX-VLM, use the `–draft-model` flag; note that DSpark in MLX-VLM currently supports greedy sampling only, so temperature should be set to zero. For llama.cpp, pair the GGUF-format drafter with the corresponding LFM2.5-VL-3B-GGUF target model.
Integration pull requests are publicly visible in the SGLang, MLX-VLM, and llama.cpp repositories. It is worth noting that acceleration of quantized models falls outside the scope of this release.
The project is released under the LFM Open License v1.0, which permits free commercial use for companies with annual revenues below $10 million. Liquid AI labels the release as experimental.
## Frequently Asked Questions
**Q: What exactly does the DSpark drafter do?**
A: It proposes a block of several tokens ahead of the main model in a single forward pass. The target model then checks the entire block at once, keeping tokens it agrees with and discarding the rest. This reduces the number of full model forward passes needed to generate a response.
**Q: Does using DSpark change the output of the model?**
A: No. Under greedy decoding, the output is identical to the base model. When using temperature-based sampling, the output distribution is mathematically preserved, meaning the statistical properties of generated text remain unchanged.
**Q: Why is the speedup different across tasks?**
A: Different tasks have different ratios of decoding time to other phases like image encoding and prefill. Tasks with longer decode phases (like image captioning) show larger relative speedups, while tasks where prefill and encoding dominate (like complex reasoning with long prompts) show smaller gains.
**Q: Can I use DSpark on quantized models?**
A: Quantized model acceleration is not covered by this release. The benchmarks were run exclusively with 16-bit weights for the vision encoder and backbone.
**Q: What block size should I use?**
A: Liquid AI recommends a block size of 8 for Apple silicon hardware and 9 for NVIDIA H100 GPUs. The choice involves a trade-off between the number of speculative tokens and the verification cost per pass.
**Q: Is this release free for commercial use?**
A: Yes, under the LFM Open License v1.0, which allows free commercial use for companies earning less than $10 million in annual revenue. The release is labeled as experimental.
**Q: Why do end-to-end speedups on edge devices lag behind data center GPUs?**
A: Edge devices have proportionally more time spent in image encoding and prefill, which speculative decoding does not accelerate. This unaccelerated portion acts as a ceiling on total speedup, consistent with Amdahl’s law.
## Conclusion
LFM2.5-VL-3B-DSpark represents a significant step forward in making vision-language models faster without compromising output quality or requiring changes to the base model architecture. By reusing the same speculative decoding framework across modalities, Liquid AI has demonstrated that a relatively small 279.5M-parameter drafter can deliver up to 3.13x faster decoding on consumer hardware and up to 2.66x on a single data center GPU.
The trade-off is clear: decode-phase speedups are substantial, but end-to-end gains are bounded by the portions of inference that speculative decoding cannot touch. For applications that are decode-bound, such as streaming text generation and real-time chat with VLMs, the improvement can be transformative. For workloads dominated by image encoding and prefill, the gains will be more modest.
With weights live on Hugging Face and immediate support across three major inference frameworks, the model is ready for developers and researchers to experiment with. As Liquid AI notes, this is an experimental release, and future iterations may address quantized acceleration and broader hardware support.
Thank you for reading
