Ordered Action Tokenization

1. The tokenizer defines what the policy has to predict.

Discrete action tokens are becoming an increasingly important design choice in modern robot learning systems: RDT-2 employs vector-quantized (VQ) action tokens for its stage-1 training; TRI's LBM/VLA leverages FAST and VQ-style tokenizations; and the winning solution of the BEHAVIOR 2025 Challenge integrated FAST tokens in training and inference.

In all of these systems, the policy sees symbols before it sees control. The tokenizer determines sequence length, the set of samples that can be decoded safely, and the left-to-right structure the model must learn. It is not a preprocessing detail; it is the prediction problem.

2. Reconstruction error is not enough.

Classical theories like the rate-distortion tradeoff focus on balancing compression rate and reconstruction fidelity. For generative robot policies, we argue that a third axis, modelability, is crucial and often overlooked: how difficult it is for a generative model to capture the distribution of a representation. Poorly structured representations may be compact and accurate, yet fundamentally hard to model.

This is the central distinction: a tokenizer can reconstruct actions well and still be poor for policy learning. If the token stream has low autoregressive modelability, or is sparse and high-entropy, the model pays that cost at every next-token prediction step.

3. A useful action tokenizer must be compact, total, and ordered.

The design target is not a single metric. An action tokenizer for autoregressive policies has to satisfy three requirements at the same time:

• (P.1) Reasonable compression. The representation should compress action chunks enough to enable efficient sequence modeling, but not so aggressively that too much information is lost.
• (P.2) Total decodability. The detokenization mapping should be a well-defined total function: every token sequence in the discrete token space must decode to a valid action chunk. This is essential because policies may generate arbitrary token sequences at inference time. If decoding is only partially defined, invalid tokens can lead to undefined behavior or catastrophic failures during execution.
• (P.3) Predictive ordering. Token sequences should admit a meaningful left-to-right causal structure aligned with next-token prediction. This structure is critical for modelability, allowing autoregressive models to learn stable, predictable token dynamics.

4. Prior tokenizers each give up one requirement.

Before OAT, each major tokenizer family misses the target in a different way. The OAT row shows the design point we want: compact, total, and ordered.

Binning. The most common scheme is per-dimension, per-timestamp binning. While simple, it scales poorly: long horizons and high-dimensional actions can produce hundreds of tokens per chunk, dramatically slowing training and inference and increasing latency. More importantly, such long, flat sequences have poor modelability across dimensions: knowing the earlier coordinates at a timestep offers little help in predicting the next one, making binning poorly aligned with autoregressive generation.

Frequency-domain transform. Frequency-based methods such as FAST achieve high information density (P.1) and impose a low-to-high frequency structure (P.3), where early tokens capture global trajectory structure and later tokens refine details. However, FAST violates P.2 (total decodability). Because Byte Pair Encoding (BPE) produces variable-length sequences, arbitrary token sequences may not decode into a valid fixed-size frequency representation, leading to undefined behavior and runtime failures. See the paper appendix and the discussion on Hugging Face for further details.

Vanilla Latents. Learned encoder-decoder latent tokenizers can achieve strong compression (P.1), and neural decoders ensure total decodability (P.2). However, the resulting token spaces often have weak autoregressive modelability: the token positions do not provide a stable left-to-right structure for next-token prediction. This makes them poorly aligned with policies that rely on meaningful left-to-right structure (P.3) for stable generation.

The gap is not that prior tokenizers are weak in every way. It is that none simultaneously make the sequence short, make the whole token space executable, and give the policy an easy left-to-right prediction problem.

5. OAT learns ordered registers that decode from any prefix.

OAT is a learned tokenizer for action chunks. It writes each chunk into a fixed set of register latents, discretizes those registers with finite scalar quantization (FSQ), and decodes generated tokens back into continuous control. The key design choice is order: register attention is causal, and nested dropout trains the decoder to reconstruct from partial prefixes.

1. Summarize the action chunk. A transformer encoder reads the continuous action sequence and writes the important temporal information into a fixed set of register tokens.
2. Discretize the registers. Finite scalar quantization turns the register latents into discrete tokens that an autoregressive policy can predict.
3. Force a left-to-right structure. Causal attention makes later registers depend on earlier registers, aligning the representation with next-token generation.
4. Train with missing tails. Nested dropout randomly masks later tokens during tokenizer training, so early tokens must carry the highest-priority information.
5. Decode back to control. A conditional decoder maps the generated token prefix back into a continuous action chunk for execution.

6. Ordered tokens turn action coding into progressive refinement.

The ordering induced by OAT admits a natural interpretation through information theory. Shannon showed that the optimal code length for an event scales with the negative logarithm of its probability, so frequent patterns require fewer bits, while rare events demand more representational capacity. Action chunks follow a similarly skewed distribution: most trajectories share common coarse structure, whereas fine-grained deviations occur infrequently.

From this perspective, OAT learns a progressive code. Early tokens capture high-probability, globally shared motion patterns; later tokens encode increasingly rare residual details. Nested dropout makes this pressure explicit: every short prefix has to reconstruct the action, so information is allocated in decreasing order of usefulness.

7. Every prefix is an executable action budget.

Because OAT trains the decoder on masked suffixes, a policy does not have to finish the token sequence before acting. A prefix can be padded, detokenized, and executed as a complete action chunk. In practice, token count becomes a runtime budget: one or two tokens for fast coarse control, more tokens when the task needs precision.

9. Token order turns representation design into a control decision.

OAT's central point is not that discrete tokens replace continuous controllers. It is that when robot policies use action tokens, token order becomes part of the control problem. Compactness, total decodability, and prefix modelability should be designed together; once they are, token count becomes a runtime budget instead of a fixed preprocessing choice.

This also changes the natural next question. In this work, autoregressive depth is fixed at deployment time. Future policies could decide depth online: simple motions may execute from short prefixes, while contact-rich steps may request more tokens before acting. That makes adaptive token budgets a concrete direction enabled by ordered, prefix-decodable action representations.

The follow-up Tokens in the Era of VLAs page takes up several of these questions directly: block-wise autoregressive generation, ordered-token supervision for VLA cotraining, and hybrid control where a continuous expert executes the final action.