Chameleon: Mixed-Modal Early-Fusion Foundation Models

16 May 2024

Papers citing "Chameleon: Mixed-Modal Early-Fusion Foundation Models"

50 / 262 papers shown

Title
FlexVAR: Flexible Visual Autoregressive Modeling without Residual Prediction Siyu Jiao Gengwei Zhang Yinlong Qian Jiancheng Huang Yao Zhao Humphrey Shi Lin Ma Y. X. Wei Zequn Jie VLM 74 2 0 27 Feb 2025
Multimodal Representation Alignment for Image Generation: Text-Image Interleaved Control Is Easier Than You Think L. Chen S. Bai Wenhao Chai Weichu Xie Haozhe Zhao Leon Vinci Junyang Lin Baobao Chang DiffM 102 6 0 27 Feb 2025
Knowledge Bridger: Towards Training-free Missing Multi-modality Completion Guanzhou Ke Shengfeng He Xinyu Wang Bo Wang Guoqing Chao Yize Zhang Yi Xie HeXing Su 115 0 0 27 Feb 2025
Chitranuvad: Adapting Multi-Lingual LLMs for Multimodal Translation Shaharukh Khan Ayush Tarun Ali Faraz Palash Kamble Vivek Dahiya Praveen Kumar Pokala Ashish Kulkarni Chandra Khatri Abhinav Ravi Shubham Agarwal 357 0 0 27 Feb 2025
Audio-FLAN: A Preliminary Release Liumeng Xue Ziya Zhou J. Pan Zhiyu Li Shuai Fan ... Haohe Liu Emmanouil Benetos Ge Zhang Yike Guo Wei Xue MLLM AuLLM CLIP VLM 65 1 0 23 Feb 2025
SAE-V: Interpreting Multimodal Models for Enhanced Alignment Hantao Lou Changye Li Yalan Qin Yaodong Yang 88 1 0 22 Feb 2025
GRAPHGPT-O: Synergistic Multimodal Comprehension and Generation on Graphs Yi Fang Bowen Jin Jiacheng Shen Sirui Ding Qiaoyu Tan Jiawei Han 134 1 0 17 Feb 2025
HermesFlow: Seamlessly Closing the Gap in Multimodal Understanding and Generation L. Yang Xinchen Zhang Ye Tian Chenming Shang Minghao Xu Wentao Zhang Tengjiao Wang 114 2 0 17 Feb 2025
LANTERN++: Enhancing Relaxed Speculative Decoding with Static Tree Drafting for Visual Auto-regressive Models Sihwan Park Doohyuk Jang Sungyub Kim Souvik Kundu Eunho Yang 105 0 0 10 Feb 2025
UniMoD: Efficient Unified Multimodal Transformers with Mixture-of-Depths Weijia Mao Zhiyong Yang Mike Zheng Shou MoE 118 0 0 10 Feb 2025
Rethinking Homogeneity of Vision and Text Tokens in Large Vision-and-Language Models Chia-Wen Kuo Sijie Zhu Fan Chen Xiaohui Shen Longyin Wen VLM 77 1 0 04 Feb 2025
MJ-VIDEO: Fine-Grained Benchmarking and Rewarding Video Preferences in Video Generation Haibo Tong Zhaoyang Wang Zhe Chen Haonian Ji Shi Qiu ... Peng Xia Mingyu Ding Rafael Rafailov Chelsea Finn Huaxiu Yao EGVM VGen 140 3 0 03 Feb 2025
Visual Generation Without Guidance Huayu Chen Kai Jiang Kaiwen Zheng Jianfei Chen Hang Su Jun Zhu 111 2 0 28 Jan 2025
VARGPT: Unified Understanding and Generation in a Visual Autoregressive Multimodal Large Language Model Xianwei Zhuang Yuxin Xie Yufan Deng Liming Liang Jinghan Ru Yuguo Yin Yuexian Zou MLLM VLM LRM 133 8 0 21 Jan 2025
Democratizing Text-to-Image Masked Generative Models with Compact Text-Aware One-Dimensional Tokens Dongwon Kim Ju He Qihang Yu Chenglin Yang Xiaohui Shen Suha Kwak Liang-Chieh Chen VLM 95 7 0 13 Jan 2025
Generative AI for Cel-Animation: A Survey Yunlong Tang Junjia Guo Pinxin Liu Zhiyuan Wang Hang Hua ... Jing Bi Mingqian Feng Xuzhao Li Zeliang Zhang Chenliang Xu VGen 113 7 0 08 Jan 2025
EditAR: Unified Conditional Generation with Autoregressive Models Jiteng Mu Nuno Vasconcelos Xinyu Wang DiffM 69 5 0 08 Jan 2025
Clinical Insights: A Comprehensive Review of Language Models in Medicine Nikita Neveditsin Pawan Lingras V. Mago LM&MA 73 5 0 08 Jan 2025
Automated Generation of Challenging Multiple-Choice Questions for Vision Language Model Evaluation Yuhui Zhang Yuchang Su Yiming Liu Xiaohan Wang James Burgess ... Josiah Aklilu Alejandro Lozano Anjiang Wei Ludwig Schmidt Serena Yeung-Levy 101 4 0 06 Jan 2025
Prepending or Cross-Attention for Speech-to-Text? An Empirical Comparison Tsz Kin Lam Marco Gaido Sara Papi L. Bentivogli Barry Haddow 81 0 0 04 Jan 2025
VisionLLM v2: An End-to-End Generalist Multimodal Large Language Model for Hundreds of Vision-Language Tasks Jiannan Wu Muyan Zhong Sen Xing Zeqiang Lai Zhaoyang Liu ... Lewei Lu Tong Lu Ping Luo Yu Qiao Jifeng Dai MLLM VLM LRM 200 51 0 03 Jan 2025
DrivingGPT: Unifying Driving World Modeling and Planning with Multi-modal Autoregressive Transformers Yuntao Chen Yuqi Wang Zhaoxiang Zhang 388 8 0 24 Dec 2024
Next Patch Prediction for Autoregressive Visual Generation Yatian Pang Peng Jin Shuo Yang Bin Lin Bin Zhu ... Liuhan Chen Francis E. H. Tay Ser-Nam Lim Harry Yang Li Yuan 180 10 0 19 Dec 2024
Parallelized Autoregressive Visual Generation Yanjie Wang Shuhuai Ren Zhijie Lin Yujin Han Haoyuan Guo Zhenheng Yang Difan Zou Jiashi Feng Xihui Liu VGen 130 12 0 19 Dec 2024
E-CAR: Efficient Continuous Autoregressive Image Generation via Multistage Modeling Zhihang Yuan Yuzhang Shang Hao Zhang Tongcheng Fang Rui Xie Bingxin Xu Yan Yan Shengen Yan Guohao Dai Yu Wang DiffM 116 1 0 18 Dec 2024
Performance Gap in Entity Knowledge Extraction Across Modalities in Vision Language Models Ido Cohen Daniela Gottesman Mor Geva Raja Giryes VLM 137 0 1 18 Dec 2024
IDEA-Bench: How Far are Generative Models from Professional Designing? C. Liang Lianghua Huang Jingwu Fang Huanzhang Dou Wei Wang Zhi-Fan Wu Yupeng Shi Junge Zhang Xin Zhao Yu Liu 3DV 103 1 0 16 Dec 2024
Towards Unified Benchmark and Models for Multi-Modal Perceptual Metrics Sara Ghazanfari Siddharth Garg Nicolas Flammarion Prashanth Krishnamurthy Farshad Khorrami Francesco Croce VLM 127 0 0 13 Dec 2024
Owl-1: Omni World Model for Consistent Long Video Generation Yuanhui Huang Wenzhao Zheng Yuan Gao Xin Tao Pengfei Wan Di Zhang Jie Zhou Jiwen Lu VGen VLM 140 0 0 12 Dec 2024
Olympus: A Universal Task Router for Computer Vision Tasks Yuanze Lin Yunsheng Li Dongdong Chen Weijian Xu Ronald Clark Philip Torr VLM ObjD 412 0 0 12 Dec 2024
[MASK] is All You Need Vincent Tao Hu Bjorn Ommer DiffM 167 5 0 09 Dec 2024
MuMu-LLaMA: Multi-modal Music Understanding and Generation via Large Language Models Shansong Liu Atin Sakkeer Hussain Qilong Wu Chenshuo Sun Ying Shan AuLLM 100 3 0 09 Dec 2024
Diffusion-VLA: Generalizable and Interpretable Robot Foundation Model via Self-Generated Reasoning Junjie Wen Minjie Zhu Yinlin Zhu Zhibin Tang Jinming Li ... Chengmeng Li Xiaoyu Liu Chaomin Shen Yaxin Peng Feifei Feng 117 13 0 04 Dec 2024
RandAR: Decoder-only Autoregressive Visual Generation in Random Orders Ziqi Pang Tianyuan Zhang Fujun Luan Yunze Man Hao Tan Kai Zhang William T. Freeman Yu-Xiong Wang VGen 111 16 0 02 Dec 2024
Unleashing In-context Learning of Autoregressive Models for Few-shot Image Manipulation Bolin Lai F. Xu Miao Liu Xiaoliang Dai Nikhil Mehta ... Zeyi Huang James M. Rehg Sangmin Lee Ning Zhang Tong Xiao 98 3 0 02 Dec 2024
OmniFlow: Any-to-Any Generation with Multi-Modal Rectified Flows Shufan Li Konstantinos Kallidromitis Akash Gokul Zichun Liao Yusuke Kato Kazuki Kozuka Aditya Grover VGen 130 5 0 02 Dec 2024
On Domain-Specific Post-Training for Multimodal Large Language Models Daixuan Cheng Shaohan Huang Ziyu Zhu Xintong Zhang Wayne Xin Zhao Zhongzhi Luan Bo Dai Zhenliang Zhang VLM 118 3 0 29 Nov 2024
Orthus: Autoregressive Interleaved Image-Text Generation with Modality-Specific Heads Siqi Kou Jiachun Jin Chang Liu Ye Ma Jian Jia Quan Chen Peng Jiang Zhijie Deng Zhijie Deng DiffM VGen VLM 176 9 0 28 Nov 2024
One Diffusion to Generate Them All Duong H. Le Tuan Pham Sangho Lee Christopher Clark Aniruddha Kembhavi Stephan Mandt Ranjay Krishna Jiasen Lu VLM 100 7 0 25 Nov 2024
Efficient Online Inference of Vision Transformers by Training-Free Tokenization Leonidas Gee Wing Yan Li V. Sharmanska Novi Quadrianto ViT 140 0 0 23 Nov 2024
Panther: Illuminate the Sight of Multimodal LLMs with Instruction-Guided Visual Prompts Honglin Li Yuting Gao Chenglu Zhu Jingdong Chen M. Yang Lin Yang MLLM 159 0 0 21 Nov 2024
Enhancing the Reasoning Ability of Multimodal Large Language Models via Mixed Preference Optimization Weiyun Wang Zhe Chen Wenhai Wang Yue Cao Yangzhou Liu ... Jinguo Zhu X. Zhu Lewei Lu Yu Qiao Jifeng Dai LRM 92 65 1 15 Nov 2024
Jailbreak Attacks and Defenses against Multimodal Generative Models: A Survey Xuannan Liu Xing Cui Peipei Li Zekun Li Huaibo Huang Shuhan Xia Miaoxuan Zhang Yueying Zou Ran He AAML 79 8 0 14 Nov 2024
Autoregressive Models in Vision: A Survey Jing Xiong Gongye Liu Lun Huang Chengyue Wu Taiqiang Wu ... Hao Fei Guillermo Sapiro Jiebo Luo Ping Luo Ngai Wong VGen 118 11 0 08 Nov 2024
Analyzing The Language of Visual Tokens David M. Chan Rodolfo Corona J. S. Park Cheol Jun Cho Yutong Bai Trevor Darrell 31 2 0 07 Nov 2024
Clustering in Causal Attention Masking Nikita Karagodin Yury Polyanskiy Philippe Rigollet 91 7 0 07 Nov 2024
The Semantic Hub Hypothesis: Language Models Share Semantic Representations Across Languages and Modalities Zhaofeng Wu Xinyan Velocity Yu Dani Yogatama Jiasen Lu Yoon Kim AIFin 70 17 0 07 Nov 2024
Addressing Representation Collapse in Vector Quantized Models with One Linear Layer Yongxin Zhu Bing Li Yifei Xin Linli Xu 65 12 0 04 Nov 2024
Randomized Autoregressive Visual Generation Qihang Yu Ju He XueQing Deng Xiaohui Shen Liang-Chieh Chen VGen DiffM 91 33 1 01 Nov 2024
Replace-then-Perturb: Targeted Adversarial Attacks With Visual Reasoning for Vision-Language Models Jonggyu Jang Hyeonsu Lyu Jungyeon Koh H. Yang VLM AAML 57 0 0 01 Nov 2024