Ordered Action Tokenization

Five terms used throughout this page.

Action chunk

A short future sequence of robot actions predicted at once, then partially executed before re-planning.

Tokenization

The map from continuous actions to discrete symbols that an autoregressive policy can model.

Detokenization

The reverse map from generated symbols back to executable continuous robot actions.

Autoregressive policy

A policy that predicts the next action token conditioned on observations and previously generated tokens.

Modelability

How easy the token distribution is for a generative model to learn, sample, and use downstream.

1. Why should we care about action tokenization?

Discrete action tokens are becoming an increasingly important design choice in modern robot learning systems. Recent systems: RDT-2 employs vector-quantized (VQ) action tokens for its stage-1 training; TRI's LBM/VLA leverages FAST and VQ-style tokenizations; and the winning solution of the BEHAVIOR 2025 Challenge integrated FAST tokens in training and inference.

Across these systems, action tokenization plays a particularly critical role during pre-training, where discrete tokens provide a structured, scalable interface between high-capacity sequence models and continuous robot control. As a result, the choice of action tokenizer increasingly shapes not only efficiency, but also what kinds of behaviors models can learn and generalize.

2. An overlooked axis: Modelability.

Classical theories like the rate-distortion tradeoff focus on balancing compression rate and reconstruction fidelity. In the era of GenAI, we argue that a third axis, modelability, is crucial and often overlooked: how difficult it is for a generative model to capture the distribution of a representation. Poorly structured representations may be compact and accurate, yet fundamentally hard to model.

This is the central distinction: a tokenizer can reconstruct actions well and still be a poor interface for policy learning. If the token stream has low autoregressive modelability, or is sparse and high-entropy, the model pays that cost at every next-token prediction step. The representation is not merely a storage format; it is the learning problem the policy actually sees.

3. Three properties we seek.

We argue that an effective action tokenizer for autoregressive policies should satisfy three key properties:

• (P.1) Reasonable compression. The representation should compress action chunks enough to enable efficient sequence modeling, but not so aggressively that too much information is lost.
• (P.2) Total decodability. The detokenization mapping should be a well-defined total function: every token sequence in the discrete token space must decode to a valid action chunk. This is essential because policies may generate arbitrary token sequences at inference time. If decoding is only partially defined, invalid tokens can lead to undefined behavior or catastrophic failures during execution.
• (P.3) Predictive ordering. Token sequences should admit a meaningful left-to-right causal structure aligned with next-token prediction. This structure is critical for modelability, allowing autoregressive models to learn stable, predictable token dynamics.

In the remainder of this article, I explain them.

4. What's missing in today's action tokens?

Each existing tokenizer family misses the target in a different way. The practical failure mode depends on which desideratum it gives up.

Binning. The most common scheme is per-dimension, per-timestamp binning. While simple, it scales poorly: long horizons and high-dimensional actions can produce hundreds of tokens per chunk, dramatically slowing training and inference and increasing latency. More importantly, such long, flat sequences have poor modelability across dimensions: knowing a[t, 1...i] offers little help in predicting a[t, i+1], making binning poorly aligned with autoregressive generation.

Frequency-domain transform. Frequency-based methods such as FAST achieve high information density (P.1) and impose a low-to-high frequency structure (P.3), where early tokens capture global trajectory structure and later tokens refine details. However, FAST violates P.2 (total decodability). Because Byte Pair Encoding (BPE) produces variable-length sequences, arbitrary token sequences may not decode into a valid fixed-size frequency representation, leading to undefined behavior and runtime failures. We refer readers to appendix of our paper and the discussion on Hugging Face for further details.

Vanilla Latents. Learned encoder-decoder latent tokenizers can achieve strong compression (P.1), and neural decoders ensure total decodability (P.2). However, the resulting token spaces often have weak autoregressive modelability: the token positions do not provide a stable left-to-right structure for next-token prediction. This makes them poorly aligned with policies that rely on meaningful left-to-right structure (P.3) for stable generation.

In summary, existing approaches each satisfy subsets of the desiderata, but none simultaneously achieve compression, total decodability, and high autoregressive modelability.

5. Ordered Action Tokenization

We introduce OAT, a learned autoencoder framework that discretizes action chunks into an ordered sequence of tokens. OAT encodes actions using transformer-based register tokens, discretizes the resulting latents with FSQ, and reconstructs actions via a conditional decoder. To improve autoregressive modelability, we combine causal attention over register tokens with nested dropout during training. Together, these design choices encourage a high-modelability latent representation in which earlier tokens capture coarse, global structure and later tokens refine details.

1. Summarize the action chunk. A transformer encoder reads the continuous action sequence and writes the important temporal information into a fixed set of register tokens.
2. Discretize the registers. Finite scalar quantization turns the register latents into discrete tokens that an autoregressive policy can predict.
3. Force a left-to-right structure. Causal attention makes later registers depend on earlier registers, aligning the representation with next-token generation.
4. Train with missing tails. Nested dropout randomly masks later tokens during tokenizer training, so early tokens must carry the highest-priority information.
5. Decode back to control. A conditional decoder maps the generated token prefix back into a continuous action chunk for execution.

6. Why does order improve modelability?

The ordering induced by OAT admits a natural interpretation through information theory. Shannon showed that the optimal code length for an event scales with the negative logarithm of its probability, - log p: frequent patterns require fewer bits, while rare events demand more representational capacity. Action chunks follow a similarly skewed distribution, most trajectories share common coarse structure, whereas fine-grained deviations occur infrequently.

From this perspective, OAT learns a form of progressive coding. Early tokens capture high-probability, globally shared motion patterns, while later tokens encode increasingly rare residual details. This ordering emerges naturally from nested dropout: because the decoder must reconstruct actions from partial prefixes, the tokenizer is incentivized to allocate information in decreasing order of frequency and importance. As a result, longer prefixes yield monotonic reconstruction improvement, and token order aligns closely with autoregressive next-token prediction, without manually assigning physical features to particular token positions.

7. A by-product: prefix-based decoding.

Autoregressive policy trained on OAT need not proceed to completion. Because any prefix of OAT token sequence can be detokenized into a valid action chunk, OAT supports prefix-based execution and enables an anytime trade-off between computation and performance. Short prefixes yield fast but coarse predictions, while longer prefixes produce more refined actions at higher computational cost. This flexibility arises naturally from the ordered tokenization and requires no changes to the policy architecture or training objective, distinguishing OAT from prior tokenizers that rely on fixed-length detokenization.

9. Closing thoughts and open directions.

A recurring question throughout this project has been whether action tokens are still necessary in the presence of powerful continuous models such as flow or diffusion policies. Our view is that future robotic systems will likely combine both discrete and continuous representations rather than choosing one over the other. A concrete example is the BEHAVIOR 2025 Challenge winning solution, which integrates discrete action tokens with continuous action experts.

A key capability enabled by OAT is prefix-based detokenization: actions can be decoded from variable-length token prefixes, yielding an anytime trade-off between computation and action fidelity. In this work, the autoregressive depth is fixed at deployment time. From an information-theoretic perspective, this is suboptimal. The number of tokens required to represent an action chunk should depend on its intrinsic complexity and the precision required for successful execution. Simple, predictable behaviors may admit compact representations, whereas complex, contact-rich interactions may require deeper autoregressive refinement. Estimating this action complexity online, and determining when additional tokens meaningfully reduce uncertainty, remains an open problem. We view adaptive autoregressive depth as a natural and important direction for future work, made possible precisely by OAT's ordered and prefix-decodable structure. Ultimately, we believe this estimation problem deserves a principled solution grounded in uncertainty and information, rather than ad hoc engineering heuristics.