GLM-5’s Training Speed Hacks: DSA and Async RL

What It Is

GLM-5 achieves competitive performance through three architectural optimizations that reduce training time and computational overhead. Dual-Stage Attention (DSA) modifies how the model processes long sequences by splitting attention computation into two phases - a coarse-grained stage that identifies relevant context regions, followed by fine-grained attention within those regions. This approach maintains long-context capabilities while reducing the quadratic complexity typical of standard attention mechanisms.

The asynchronous reinforcement learning setup separates token generation from gradient computation. Traditional RL training for language models generates text, evaluates it, then updates weights in a sequential pipeline. GLM-5 decouples these operations so generation happens on separate workers while training continues on existing batches. The architecture includes dedicated inference nodes that produce rollouts while training nodes consume completed episodes from a queue.

Agent RL algorithms extend beyond single-turn responses to multi-step reasoning tasks. Rather than optimizing for immediate reward signals, these methods credit actions based on their contribution to eventual task completion. For coding benchmarks where solutions require multiple function calls or debugging iterations, this approach proves more effective than standard RLHF techniques.

Why It Matters

Training efficiency directly impacts which organizations can develop competitive models. DSA reduces the memory footprint for processing 128K token contexts by approximately 40% compared to full attention, making long-context training feasible on smaller GPU clusters. Research teams without access to massive compute budgets can experiment with similar context windows previously reserved for well-funded labs.

The asynchronous RL architecture addresses a bottleneck in post-training workflows. Generating rollouts for policy optimization typically consumes 60-70% of wall-clock time in synchronous setups. By parallelizing generation and training, GLM-5’s approach cuts post-training time nearly in half. This matters for rapid iteration cycles - teams can test more reward formulations and hyperparameter configurations within fixed compute budgets.

Coding benchmarks show the practical impact. GLM-5 outperforms other open-source models on HumanEval and MBPP, suggesting these optimizations translate to measurable capability improvements rather than just faster training. The agent RL component appears particularly relevant for tool-use scenarios where models must chain multiple API calls or debug failed attempts.

Getting Started

The technical details appear in the GLM-5 paper at https://arxiv.org/abs/2602.15763, which includes pseudocode for the asynchronous training loop. Implementing DSA requires modifying the attention layer to include a routing mechanism:

# Simplified DSA concept def dual_stage_attention(query, key, value, chunk_size=512):
 # Stage 1: Coarse attention over chunks
 chunk_scores = compute_chunk_relevance(query, key, chunk_size)
 top_chunks = select_top_k_chunks(chunk_scores, k=8)
 
 # Stage 2: Fine-grained attention within selected chunks
 attended_values = []
 for chunk_idx in top_chunks:
 chunk_attn = standard_attention(
 query, 
 key[chunk_idx], 
 value[chunk_idx]
 )
 attended_values.append(chunk_attn)
 
 return combine_chunk_outputs(attended_values)

For asynchronous RL, the architecture requires separate inference and training clusters communicating through a shared buffer. Inference workers run the current policy to generate rollouts, while training workers consume batches from the buffer to compute policy gradients. This setup demands careful synchronization to prevent policy lag between generation and training.

Context

DSA resembles other sparse attention patterns like Longformer’s sliding window or BigBird’s random attention, but focuses specifically on training efficiency rather than inference speed. The two-stage approach trades some theoretical expressiveness for practical gains - models can’t attend to every token, but empirical results suggest the selected chunks capture sufficient context.

Alternative async RL implementations exist in robotics and game-playing domains, where IMPALA and Ape-X pioneered similar decoupled architectures. Applying these techniques to language model post-training represents a relatively recent development. The main limitation involves staleness - training on rollouts from slightly outdated policies can destabilize learning if the policy diverges too quickly.

Compared to distillation-based speedups or quantization, these methods reduce training cost rather than inference cost. Teams prioritizing deployment efficiency might find greater value in techniques like speculative decoding or structured pruning. However, for organizations focused on developing new base models, GLM-5’s training optimizations offer a practical blueprint for resource-constrained research.