Cluster-Oriented Token Prediction for Autoregressive Visual Generation (IAR)
A method that improves LLM-style autoregressive image generation, where images are generated as sequences of discrete tokens from a vector-quantized codebook. Exploiting the strong correlation among visual token embeddings, it (1) applies balanced k-means Codebook Rearrangement so visually similar tokens share a cluster, and (2) adds a Cluster-oriented Cross-entropy Loss that rewards predicting the correct cluster, so a mispredicted token still tends to be visually similar to the target. This halves training time at equal FID from 100M to 1.4B parameters and obeys scaling laws.
Improving Autoregressive Visual Generation with Cluster-Oriented Token Prediction Teng Hu1∗
IAR (Improved AutoRegressive visual generation) improves LLM-style autoregressive image generation, in which an image is produced as a sequence of discrete visual tokens drawn from a learned vector-q…