Transfusion: Predict the Next Token and Diffuse Images with One Multi-Modal Model

20 August 2024

Lili Yu

Papers citing "Transfusion: Predict the Next Token and Diffuse Images with One Multi-Modal Model"

50 / 123 papers shown

Title
Piece it Together: Part-Based Concepting with IP-Priors Elad Richardson Kfir Goldberg Yuval Alaluf Daniel Cohen-Or DiffM 71 0 0 13 Mar 2025
OmniMamba: Efficient and Unified Multimodal Understanding and Generation via State Space Models Jialv Zou Bencheng Liao Qian Zhang Wenyu Liu Xinggang Wang Mamba MLLM 82 1 0 11 Mar 2025
3D Point Cloud Generation via Autoregressive Up-sampling Ziqiao Meng Qichao Wang Zhipeng Zhou Irwin King Peilin Zhao 3DPC 66 0 0 11 Mar 2025
A Survey on Knowledge-Oriented Retrieval-Augmented Generation Mingyue Cheng Yucong Luo Jie Ouyang Qiang Liu Huijie Liu ... Bohou Zhang Jiawei Cao Jie Ma Daoyu Wang Enhong Chen 3DV 78 3 0 11 Mar 2025
LMM-R1: Empowering 3B LMMs with Strong Reasoning Abilities Through Two-Stage Rule-Based RL Yingzhe Peng Gongrui Zhang Miaosen Zhang Zhiyuan You Jie Liu Qipeng Zhu Kai Yang Xingzhong Xu Xin Geng Xu Yang LRM ReLM 105 36 0 10 Mar 2025
WISE: A World Knowledge-Informed Semantic Evaluation for Text-to-Image Generation Yuwei Niu Munan Ning Mengren Zheng Bin Lin Peng Jin Jiaqi Liao Kunpeng Ning Bin Zhu Li Yuan EGVM 72 15 0 10 Mar 2025
Unleashing the Potential of Large Language Models for Text-to-Image Generation through Autoregressive Representation Alignment Xing Xie Jiawei Liu Ziyue Lin Huijie Fan Zhi Han Yandong Tang Liangqiong Qu 47 0 0 10 Mar 2025
V2Flow: Unifying Visual Tokenization and Large Language Model Vocabularies for Autoregressive Image Generation Guiwei Zhang Tianyu Zhang Mohan Zhou Yalong Bai Biye Li 74 0 0 10 Mar 2025
Should VLMs be Pre-trained with Image Data? Sedrick Scott Keh Jean Mercat S. Gadre Kushal Arora Igor Vasiljevic ... Shuran Song Russ Tedrake Thomas Kollar Ludwig Schmidt Achal Dave VLM 54 0 0 10 Mar 2025
SemHiTok: A Unified Image Tokenizer via Semantic-Guided Hierarchical Codebook for Multimodal Understanding and Generation Zhenpeng Chen Chunwei Wang Xiuwei Chen Hang Xu Jiawei Han Xiandan Liang VLM 76 1 0 09 Mar 2025
Unlocking Pretrained LLMs for Motion-Related Multimodal Generation: A Fine-Tuning Approach to Unify Diffusion and Next-Token Prediction Shinichi Tanaka Zhao Wang Yoichi Kato Jun Ohya DiffM 56 0 0 08 Mar 2025
Frequency Autoregressive Image Generation with Continuous Tokens Hu Yu Hao Luo Hangjie Yuan Yu Rong Feng Zhao VGen 54 3 0 07 Mar 2025
DoraCycle: Domain-Oriented Adaptation of Unified Generative Model in Multimodal Cycles Rui Zhao Weijia Mao Mike Zheng Shou 71 0 0 05 Mar 2025
MINT: Multi-modal Chain of Thought in Unified Generative Models for Enhanced Image Generation Yi Wang Mushui Liu Wanggui He Longxiang Zhang Z. Huang ... Haoyang Li Weilong Dai Mingli Song Jie Song Hao Jiang MLLM MoE LRM 97 1 0 03 Mar 2025
Direct Discriminative Optimization: Your Likelihood-Based Visual Generative Model is Secretly a GAN Discriminator Kaiwen Zheng Yongxin Chen Huayu Chen Guande He Xuan Li Jun Zhu Qinsheng Zhang DiffM 49 0 0 03 Mar 2025
FlexVAR: Flexible Visual Autoregressive Modeling without Residual Prediction Siyu Jiao Gengwei Zhang Yinlong Qian Jiancheng Huang Yao Zhao Humphrey Shi Lin Ma Y. X. Wei Zequn Jie VLM 51 2 0 27 Feb 2025
HermesFlow: Seamlessly Closing the Gap in Multimodal Understanding and Generation L. Yang Xinchen Zhang Ye Tian Chenming Shang Minghao Xu Wentao Zhang Bin Cui 102 1 0 17 Feb 2025
UniMoD: Efficient Unified Multimodal Transformers with Mixture-of-Depths Weijia Mao Zheng Yang Mike Zheng Shou MoE 85 0 0 10 Feb 2025
Generative AI for Cel-Animation: A Survey Yunlong Tang Junjia Guo Pinxin Liu Zhiyuan Wang Hang Hua ... Jing Bi Mingqian Feng Xuzhao Li Zeliang Zhang Chenliang Xu VGen 96 8 0 08 Jan 2025
EditAR: Unified Conditional Generation with Autoregressive Models Jiteng Mu Nuno Vasconcelos Xinyu Wang DiffM 47 5 0 08 Jan 2025
CAT: Content-Adaptive Image Tokenization Junhong Shen Kushal Tirumala Michihiro Yasunaga Ishan Misra Luke Zettlemoyer Lili Yu Chunting Zhou 50 0 0 06 Jan 2025
HoVLE: Unleashing the Power of Monolithic Vision-Language Models with Holistic Vision-Language Embedding Chenxin Tao Shiqian Su X. Zhu Chenyu Zhang Zhe Chen ... Wenhai Wang Lewei Lu Gao Huang Yu Qiao Jifeng Dai MLLM VLM 115 2 0 20 Dec 2024
Next Patch Prediction for Autoregressive Visual Generation Yatian Pang Peng Jin Shuo Yang Bin Lin Bin Zhu ... Liuhan Chen Francis E. H. Tay Ser-Nam Lim Harry Yang Li Yuan 145 9 0 19 Dec 2024
E-CAR: Efficient Continuous Autoregressive Image Generation via Multistage Modeling Zhihang Yuan Yuzhang Shang Hao Zhang Tongcheng Fang Rui Xie Bingxin Xu Yan Yan Shengen Yan Guohao Dai Yu Wang DiffM 108 1 0 18 Dec 2024
SoftVQ-VAE: Efficient 1-Dimensional Continuous Tokenizer Hongyu Chen Zihan Wang Xianrui Li Xingwu Sun Fangyi Chen Jiang Liu Rongxiang Weng Bhiksha Raj Zicheng Liu Emad Barsoum VLM 116 7 0 14 Dec 2024
Olympus: A Universal Task Router for Computer Vision Tasks Yuanze Lin Yunsheng Li Dongdong Chen Weijian Xu Ronald Clark Philip Torr VLM ObjD 308 0 0 12 Dec 2024
UniReal: Universal Image Generation and Editing via Learning Real-world Dynamics Xi Chen Zhifei Zhang He Zhang Yuqian Zhou Seunggeun Kim ... Nanxuan Zhao Yilin Wang Hui Ding Zhe Lin Hengshuang Zhao VGen DiffM 129 21 0 10 Dec 2024
From Slow Bidirectional to Fast Autoregressive Video Diffusion Models Tianwei Yin Qiang Zhang Richard Zhang William T. Freeman F. Durand Eli Shechtman Xun Huang VGen DiffM 101 5 0 10 Dec 2024
[MASK] is All You Need Vincent Tao Hu Bjorn Ommer DiffM 148 3 0 09 Dec 2024
MuMu-LLaMA: Multi-modal Music Understanding and Generation via Large Language Models Shansong Liu Atin Sakkeer Hussain Qilong Wu Chenshuo Sun Ying Shan AuLLM 77 3 0 09 Dec 2024
Diffusion-VLA: Scaling Robot Foundation Models via Unified Diffusion and Autoregression Junjie Wen Minjie Zhu Bo Li Zhibin Tang Jinming Li ... Chengmeng Li Xiaoyu Liu Chaomin Shen Yaxin Peng Feifei Feng 101 17 0 04 Dec 2024
RandAR: Decoder-only Autoregressive Visual Generation in Random Orders Ziqi Pang Tianyuan Zhang Fujun Luan Yunze Man Hao Tan Kai Zhang William T. Freeman Yu-Xiong Wang VGen 94 15 0 02 Dec 2024
XQ-GAN: An Open-source Image Tokenization Framework for Autoregressive Generation Xianrui Li Kai Qiu Hongyu Chen Jason Kuen Jiuxiang Gu Rongxiang Weng Zhe Lin Bhiksha Raj VLM 137 3 0 02 Dec 2024
Unleashing In-context Learning of Autoregressive Models for Few-shot Image Manipulation Bolin Lai F. Xu Miao Liu Xiaoliang Dai Nikhil Mehta ... Zeyi Huang James M. Rehg Sangmin Lee Ning Zhang Tong Xiao 86 2 0 02 Dec 2024
OmniFlow: Any-to-Any Generation with Multi-Modal Rectified Flows Shufan Li Konstantinos Kallidromitis Akash Gokul Zichun Liao Yusuke Kato Kazuki Kozuka Aditya Grover VGen 108 5 0 02 Dec 2024
Puzzle: Distillation-Based NAS for Inference-Optimized LLMs Akhiad Bercovich Tomer Ronen Talor Abramovich Nir Ailon Nave Assaf ... Ido Shahaf Oren Tropp Omer Ullman Argov Ran Zilberstein Ran El-Yaniv 94 1 0 28 Nov 2024
Orthus: Autoregressive Interleaved Image-Text Generation with Modality-Specific Heads Siqi Kou Jiachun Jin Chang Liu Ye Ma Jian Jia Quan Chen Peng Jiang Zhijie Deng Zhijie Deng DiffM VGen VLM 137 7 0 28 Nov 2024
Collaborative Decoding Makes Visual Auto-Regressive Modeling Efficient Zigeng Chen Xinyin Ma Gongfan Fang Xinchao Wang VLM 91 5 0 26 Nov 2024
One Diffusion to Generate Them All Duong H. Le Tuan Pham Sangho Lee Christopher Clark Aniruddha Kembhavi Stephan Mandt Ranjay Krishna Jiasen Lu VLM 84 5 0 25 Nov 2024
Artificial Intelligence for Biomedical Video Generation Linyuan Li Jianing Qiu Anujit Saha Lin Li Poyuan Li Mengxian He Ziyu Guo Wu Yuan VGen 70 1 0 12 Nov 2024
Autoregressive Models in Vision: A Survey Jing Xiong Gongye Liu Lun Huang Chengyue Wu Taiqiang Wu ... Hao Fei Guillermo Sapiro Jiebo Luo Ping Luo Ngai Wong VGen 53 9 0 08 Nov 2024
Analyzing The Language of Visual Tokens David M. Chan Rodolfo Corona J. S. Park Cheol Jun Cho Yutong Bai Trevor Darrell 31 2 0 07 Nov 2024
GameGen-X: Interactive Open-world Game Video Generation Haoxuan Che Xuanhua He Quande Liu Cheng Jin Hao Chen VGen 71 17 0 01 Nov 2024
Towards Unifying Understanding and Generation in the Era of Vision Foundation Models: A Survey from the Autoregression Perspective Shenghao Xie Wenqiang Zu Mingyang Zhao Duo Su Shilong Liu Ruohua Shi Guoqi Li Shanghang Zhang Lei Ma LRM 58 3 0 29 Oct 2024
Unbounded: A Generative Infinite Game of Character Life Simulation Jialu Li Yuanzhen Li Neal Wadhwa Yael Pritch David E. Jacobs Michael Rubinstein Joey Tianyi Zhou Nataniel Ruiz VGen AI4CE 46 4 0 24 Oct 2024
Infinity-MM: Scaling Multimodal Performance with Large-Scale and High-Quality Instruction Data Shuhao Gu Jialing Zhang Siyuan Zhou Kevin Yu Zhaohu Xing ... Yufeng Cui Xinlong Wang Yaoqi Liu Fangxiang Feng Guang Liu SyDa VLM MLLM 34 21 0 24 Oct 2024
Fluid: Scaling Autoregressive Text-to-image Generative Models with Continuous Tokens Lijie Fan Tianhong Li Siyang Qin Yuanzhen Li Chen Sun Michael Rubinstein Deqing Sun Kaiming He Yonglong Tian VLM DiffM 53 43 0 17 Oct 2024
PUMA: Empowering Unified MLLM with Multi-granular Visual Generation Rongyao Fang Chengqi Duan Kun Wang Hao Li H. Tian Xingyu Zeng Rui Zhao Jifeng Dai Hongsheng Li Xihui Liu MLLM 41 11 0 17 Oct 2024
$γ-$ MoD: Exploring Mixture-of-Depth Adaptation for Multimodal Large Language Models Yaxin Luo Gen Luo Jiayi Ji Yiyi Zhou Xiaoshuai Sun Zhiqiang Shen Rongrong Ji VLM MoE 54 1 0 17 Oct 2024
Janus: Decoupling Visual Encoding for Unified Multimodal Understanding and Generation Chengyue Wu Xiaokang Chen Z. F. Wu Yiyang Ma Xingchao Liu ... Wen Liu Zhenda Xie Xingkai Yu Chong Ruan Ping Luo AI4TS 65 82 0 17 Oct 2024