HOFAR: High-Order Augmentation of Flow Autoregressive Transformers

Flow Matching and Transformer architectures have demonstrated remarkable performance in image generation tasks, with recent work FlowAR [Ren et al., 2024] synergistically integrating both paradigms to advance synthesis fidelity. However, current FlowAR implementations remain constrained by first-order trajectory modeling during the generation process. This paper introduces a novel framework that systematically enhances flow autoregressive transformers through high-order supervision. We provide theoretical analysis and empirical evaluation showing that our High-Order FlowAR (HOFAR) demonstrates measurable improvements in generation quality compared to baseline models. The proposed approach advances the understanding of flow-based autoregressive modeling by introducing a systematic framework for analyzing trajectory dynamics through high-order expansion.

@article{liang2025_2503.08032,
  title={ HOFAR: High-Order Augmentation of Flow Autoregressive Transformers },
  author={ Yingyu Liang and Zhizhou Sha and Zhenmei Shi and Zhao Song and Mingda Wan },
  journal={arXiv preprint arXiv:2503.08032},
  year={ 2025 }
}