NVIDIA has released Nemotron-Labs-TwoTower, an open-weight diffusion language model that addresses a fundamental throughput bottleneck in autoregressive text generation. Built on the Nemotron-3-Nano-30B-A3B backbone, the model replaces the serial token-by-token decoding of conventional large language models with a block-wise parallel diffusion process, achieving 2.42× wall-clock generation throughput while retaining 98.7 percent of the autoregressive baseline’s aggregate benchmark quality. The weights ship under the NVIDIA Nemotron Open Model License, making the architecture available for commercial use and further research.
TL;DR
- TwoTower separates diffusion into a frozen autoregressive context tower and a trained denoiser tower.
- It retains 98.7 percent of autoregressive quality at 2.42× throughput at the default operating point (γ=0.8, S=16, 2×H100).
- The denoiser was trained on approximately 2.1 trillion tokens; the backbone used 25 trillion.
- A single checkpoint supports diffusion, mock-autoregressive, and full autoregressive decoding modes.
How the Two-Tower Architecture Works
Autoregressive models generate one token at a time, a serial process that caps throughput. Discrete diffusion language models take a different approach: they generate tokens in parallel and refine them iteratively. Most diffusion models use a single network to represent clean tokens and denoise corrupted ones at every step. TwoTower separates these responsibilities into two distinct networks, or towers.
The autoregressive context tower runs causally over the prompt and committed tokens, producing per-layer KV cache and final Mamba-2 states. This tower remains frozen and preserves the backbone’s autoregressive capability. The diffusion denoiser tower refines noisy blocks using bidirectional in-block attention while staying causal with respect to past clean blocks. The towers connect layer by layer: denoiser layer i cross-attends to context tower layer i, providing multi-scale access to the backbone’s representations rather than broadcasting only the last hidden state as prior approaches have done.
Each tower contains 52 layers: 23 Mamba-2, 6 self-attention, and 23 mixture-of-experts layers. The released checkpoint ships both towers at roughly 60 billion total parameters, with approximately 3 billion active parameters per tower per token. The MoE uses 128 routable experts, of which six activate, plus two shared experts. The denoiser was trained on approximately 2.1 trillion tokens, a fraction of the backbone’s 25-trillion-token pretraining corpus.
Benchmark Performance and Throughput Gains
Evaluations were conducted in BF16 on two H100 GPUs. The default operating point uses confidence unmasking with threshold γ=0.8 and block size S=16. Across a broad suite of benchmarks, TwoTower diffusion decoding preserves quality within about one point of the autoregressive baseline on general knowledge tasks, with modest degradation in code and math and slight improvements in commonsense and multilingual scores.
On MMLU (5-shot accuracy), the autoregressive baseline scores 78.56 against TwoTower’s 78.24. ARC-Challenge (25-shot normalized accuracy) shows a slight improvement: 91.72 versus 92.66. Code benchmarks show more noticeable degradation, with HumanEval dropping from 79.27 to 75.58 and MATH-500 from 84.40 to 80.60. The aggregate quality retained across all evaluation tasks stands at 98.7 percent, while generation throughput reaches 2.42× that of the autoregressive baseline.
Three Generation Modes in One Checkpoint
The checkpoint exposes three inference paths. Two-tower diffusion uses two GPUs with approximately 59 GB per GPU in BF16. The first generation mode, generate_mask_diffusion, commits up to block_size tokens per step using confidence-based unmasking. The second mode, generate_mock_ar, commits one token per step while still using the diffusion denoiser. The third mode, generate_ar, runs the autoregressive backbone alone on a single 80 GB GPU. This means teams can evaluate and deploy all three decoding strategies from a single checkpoint without maintaining separate model copies.
Generation proceeds block by block. Each block starts as S [MASK] tokens, the denoiser refines it over T steps, and then commits the result. The context tower processes committed tokens to update its caches for the next block. Because multiple tokens are committed early in refinement, the approach can beat one-token-per-step autoregressive decoding even when multiple denoising steps are used per block.
Use Cases and Trade-Offs
The most direct application is faster batch generation for synthetic data pipelines. A team producing training text can trade a 1.3 percent quality drop for more than double the throughput at the default γ=0.8 operating point. Raising the confidence threshold preserves more quality, while lowering it commits more tokens per step for higher speed.
A second use case is tuning the quality–throughput trade-off for latency-sensitive applications. The same checkpoint can run in full diffusion mode for throughput or in autoregressive mode for maximum quality, depending on the workload. A third use case is drop-in adaptation: the context tower retains its language model head for speculative decoding, verification, or autoregressive scoring, so teams can combine both decoding paradigms without engineering overhead.
The model’s primary limitation is hardware demand. Full two-tower diffusion requires two GPUs and approximately 59 GB per GPU in BF16. Code and math tasks degrade more than general knowledge, and the released checkpoint is a base model that has not undergone instruction tuning or alignment. Throughput beyond 3× comes with larger quality loss, so the default operating point represents a practical balance.
Who Should Try This Now
Developers and research teams already working with the Nemotron-3-Nano-30B-A3B backbone can download the TwoTower checkpoint and run all three generation modes using the Hugging Face Transformers integration with trust_remote_code=True. The most practical first step is to benchmark the default diffusion configuration against an existing autoregressive pipeline on a representative generation task, measuring both throughput and output quality for the specific use case. The threshold γ and block size S provide two knobs for tuning the trade-off, and the single-checkpoint design means no infrastructure changes are required to switch between modes.
