Training AI agents to autonomously write code or navigate software systems is a compute nightmare for one primary reason: GPU idle time. When a single rollout takes hours and consumes close to a million tokens, traditional training infrastructure effectively breaks. Alibaba’s Qwen team has introduced QwenGyre, an elastic reinforcement learning framework designed specifically to solve this “XLong-horizon” problem by dynamically repurposing idle GPUs and efficiently processing the branching execution trees that modern agent harnesses produce.
The Deep-Rooted Inefficiency in Long-Horizon Agent Training
In standard RL training setups, the lifecycle of a learning step involves a rollout phase followed by a training phase. Colocation locks a set of GPUs for a single agent’s entire journey, meaning expensive hardware sits idle while the agent waits for a tool to execute. Asynchronous systems separate the pools, but a dedicated rollout pool cannot lend its GPUs to the training pool when it hits a lull, stranding capacity on the other side of the wall. At the scale of XLong-horizon tasks, where a coding agent must generate an entire repository from a single specification, these inefficiencies compound dramatically.
The elastic scheduler demonstrates that the hardware ceiling is not a fixed constraint if the software is willing to dynamically renegotiate resource boundaries.
The research team defines XLong-horizon tasks as agent runs spanning hours and hundreds of model and tool interactions, often processing around 700,000 tokens per rollout and reaching up to one million. Harnesses like Claude Code are not simple stateless APIs; they compact context dynamically, spawn sub-agents, retry failed commands, and create deeply nested, branching execution trees. Expanding every branch of this tree into a full training sample is computationally prohibitive and introduces massive duplication. QwenGyre was built from the ground up to attack both the scheduling problem and the data-engineering problem simultaneously.
QwenGyre’s Elastic Scheduler: A Cell-Based Approach to GPU Utilization
Alibaba’s team partitions GPUs into cells. A core cell holds the authoritative model weights and the optimizer state. Satellite cells can dynamically attach to the training process or detach to serve rollouts. The key innovation is a scheduler that tracks a “waterlevel”: the number of queued and running executions. When the waterlevel drops low enough that the remaining cells can handle the live workload, idle cells instantly pivot to training.
Critically, the agent harness never pauses during this transition. Model calls are rerouted through a proxy, so the agent continues executing its tool calls without interruption. This rapid reconfiguration is enabled by two technical feats: KV-cache blocks migrate over RDMA to avoid expensive prefix recomputation, and satellites pull core weights through Mooncake’s Transfer Engine to join an ongoing batch. Streaming training further allows cells to claim micro-steps as groups of data become ready, so faster cells can take on more work.
How fast is a QwenGyre role switch?
An average role switch from rollout to training takes 8.52 seconds, and a switch from training back to rollout takes 3.46 seconds. This sub-ten-second transition is fast enough to reclaim GPU cycles that would otherwise be lost in a traditional pipeline, especially during the long tail of a rollout where most agents have finished but a few are still executing.
This architecture was tested on massive hardware configurations. The Qwen 3.6 122B model runs used 24 to 32 nodes, while the Qwen 3.8 2.4T parameter model run used 192 nodes organized into four cells of 48. Megatron handles the training workload, and SGLang serves the rollouts. The result is a system that can essentially hot-swap a GPU between running an agent and learning from an agent, without dropping a single request.
The Trajectory Processor: Taming Branching Histories
Even with perfect GPU utilization, the training loop breaks if the data ingested is polluted by the branching structure of modern agent harnesses. A single Claude Code execution can spawn multiple sub-agents, retry steps, and write summaries of the results, creating a tree of interactions rather than a single linear trajectory. Naively expanding every path duplicates shared prefixes and inflates training cost without providing new signal.
QwenGyre’s Trajectory Processor solves this with a systematic approach. A proxy records exact token IDs and behavior log-probabilities using a token-in, token-out method, ensuring precise provenance of every token. Timed-out executions are still scored based on their preserved workspace state, so partial progress contributes signal instead of being discarded as a zero reward. Failures are excluded outright rather than treated as negative examples.
Bounded sampling limits the damage from over-branching. At most five trajectories per execution are kept, ranked by importance: main trajectory, main summary, sub-agent trajectories, and sub-agent summaries. Crucially, shared target prefixes are counted only once, and only tokens generated by the policy are trained—system prompts and tool outputs remain context only. Losses are averaged fairly within each execution, so an agent that happens to branch heavily does not inflate its weight in the gradient update. On the Qwen 3.6 122B model, this cap of five trajectories matched uncapped training scores while cutting forward-backward compute time by 25.2 percent.
Measuring the Margin: Benchmark Results
All benchmark runs used Claude Code 2.1.220 as the agent harness, with equal GPU budgets and matched model staleness between setups to ensure a fair comparison. The results show a clear, if variable, edge over both the Async and Colocate baselines.
| Workload | Speedup vs Async | Speedup vs Colocate |
|---|---|---|
| NL2RepoBench (Qwen 3.6 122B, 48 steps) | 1.42x to 1.53x | 1.36x to 1.47x |
| DeepSWE (Qwen 3.6 122B, 24 steps) | 1.57x | 1.61x to 1.82x |
| TerminalBench (Qwen 3.6 122B, 24 steps) | 1.38x to 1.43x | 1.81x to 1.85x |
| NL2RepoBench (Qwen 3.8 2.4T, 48 steps) | 1.78x | 1.21x |
The best-case speedup was 1.85x over Colocate on TerminalBench, where rollouts have naturally varied durations and the elastic scheduler can reclaim idle cycles most aggressively. The worst-case speedup was 1.21x over Colocate on the 2.4T parameter model, where many queries ran so close to the timeout limit that there was less idle time to reclaim. On this same 2.4T run, QwenGyre finished 48 training steps in 75.42 hours, compared to 134.55 hours for Async and 91.47 hours for Colocate. The evaluation pass rate on NL2RepoBench rose from 52.48 percent to 58.54 percent.
Why This Architecture Matters for the Next Wave of Agent AI
The AI industry is pivoting hard toward agentic workflows, but the training infrastructure for these systems has not kept pace. QwenGyre is a systemic response that addresses the two choke points holding back long-horizon RL: GPU utilization and trajectory data quality. The elastic scheduler demonstrates that the hardware ceiling is not a fixed constraint if the software is willing to dynamically renegotiate resource boundaries. The trajectory processor solves a specific, nasty data-engineering problem caused by agents that branch and retry in ways that standard RL pipelines simply cannot digest.
The paper links to no public code release, which is a practical disappointment for the open-source community, but the architectural patterns it describes are clear enough to inspire replication by teams building their own agent training stacks. For enterprises building coding agents, QwenGyre validates that the future of AI training is not just about better chips or larger models, but about fundamentally smarter resource orchestration and data discipline.
- What is QwenGyre?QwenGyre is an elastic reinforcement learning framework developed by Alibaba's Qwen team to reduce GPU idle time in long-horizon agent training.
- How does QwenGyre reduce GPU idle time?QwenGyre uses an elastic scheduler that partitions GPUs into cells and dynamically reallocates idle cells from rollout to training based on a waterlevel metric.
- What performance improvements does QwenGyre achieve?QwenGyre achieved a worst-case speedup of 1.21x over colocation and completed training steps significantly faster than asynchronous systems.
- Why is QwenGyre important for agent AI?QwenGyre addresses the two choke points in long-horizon RL: GPU utilization and trajectory data quality, enabling more efficient training of coding agents.
- Is QwenGyre open source?The paper does not link to a public code release, but the architectural patterns are clear enough for replication.
