VL-GPT: A Generative Pre-trained Transformer for Vision and Language Understanding and Generation

14 December 2023

Sijie Zhao

Ying Shan

Papers citing "VL-GPT: A Generative Pre-trained Transformer for Vision and Language Understanding and Generation"

28 / 28 papers shown

Title
Unified Multimodal Understanding and Generation Models: Advances, Challenges, and Opportunities Xuzhi Zhang Jintao Guo Shanshan Zhao Minghao Fu Lunhao Duan Guo-Hua Wang Qing-Guo Chen Zhao Xu Weihua Luo Kaifu Zhang DiffM 74 0 0 05 May 2025
AnimeGamer: Infinite Anime Life Simulation with Next Game State Prediction Junhao Cheng Yuying Ge Yixiao Ge Jing Liao Ying Shan VGen AI4CE 56 0 0 01 Apr 2025
Teller: Real-Time Streaming Audio-Driven Portrait Animation with Autoregressive Motion Generation Dingcheng Zhen Shunshun Yin Shiyang Qin Hou Yi Ziwei Zhang Siyuan Liu Gan Qi Ming Tao VGen 71 0 0 24 Mar 2025
UniMoD: Efficient Unified Multimodal Transformers with Mixture-of-Depths Weijia Mao Z. Yang Mike Zheng Shou MoE 78 0 0 10 Feb 2025
Visual Large Language Models for Generalized and Specialized Applications Yifan Li Zhixin Lai Wentao Bao Zhen Tan Anh Dao Kewei Sui Jiayi Shen Dong Liu Huan Liu Yu Kong VLM 88 11 0 06 Jan 2025
Olympus: A Universal Task Router for Computer Vision Tasks Yuanze Lin Yunsheng Li Dongdong Chen Weijian Xu Ronald Clark Philip H. S. Torr VLM ObjD 197 0 0 12 Dec 2024
Towards Unifying Understanding and Generation in the Era of Vision Foundation Models: A Survey from the Autoregression Perspective Shenghao Xie Wenqiang Zu Mingyang Zhao Duo Su Shilong Liu Ruohua Shi Guoqi Li Shanghang Zhang Lei Ma LRM 44 3 0 29 Oct 2024
PUMA: Empowering Unified MLLM with Multi-granular Visual Generation Rongyao Fang Chengqi Duan Kun Wang Hao Li H. Tian Xingyu Zeng Rui Zhao Jifeng Dai Hongsheng Li Xihui Liu MLLM 36 11 0 17 Oct 2024
Multi-Modal Generative AI: Multi-modal LLM, Diffusion and Beyond Hong Chen Xin Wang Yuwei Zhou Bin Huang Yipeng Zhang Wei Feng Houlun Chen Zeyang Zhang Siao Tang Wenwu Zhu DiffM 55 7 0 23 Sep 2024
Show-o: One Single Transformer to Unify Multimodal Understanding and Generation Jinheng Xie Weijia Mao Zechen Bai David Junhao Zhang Weihao Wang Kevin Qinghong Lin Yuchao Gu Zhijie Chen Zhenheng Yang Mike Zheng Shou 57 164 0 22 Aug 2024
SEED-Story: Multimodal Long Story Generation with Large Language Model Shuai Yang Yuying Ge Yang Li Yukang Chen Yixiao Ge Ying Shan Yingcong Chen VGen DiffM 75 26 0 11 Jul 2024
Lateralization LoRA: Interleaved Instruction Tuning with Modality-Specialized Adaptations Zhiyang Xu Minqian Liu Ying Shen Joy Rimchala Jiaxin Zhang Qifan Wang Yu Cheng Lifu Huang VLM 39 2 0 04 Jul 2024
OmniCorpus: A Unified Multimodal Corpus of 10 Billion-Level Images Interleaved with Text Qingyun Li Zhe Chen Weiyun Wang Wenhai Wang Shenglong Ye ... Dahua Lin Yu Qiao Botian Shi Conghui He Jifeng Dai VLM OffRL 56 20 0 12 Jun 2024
Vision Model Pre-training on Interleaved Image-Text Data via Latent Compression Learning Chenyu Yang Xizhou Zhu Jinguo Zhu Weijie Su Junjie Wang ... Lewei Lu Bin Li Jie Zhou Yu Qiao Jifeng Dai VLM CLIP 44 5 0 11 Jun 2024
Needle In A Multimodal Haystack Weiyun Wang Shuibo Zhang Yiming Ren Yuchen Duan Tiantong Li ... Ping Luo Yu Qiao Jifeng Dai Wenqi Shao Wenhai Wang VLM 59 17 0 11 Jun 2024
The Evolution of Multimodal Model Architectures S. Wadekar Abhishek Chaurasia Aman Chadha Eugenio Culurciello 43 14 0 28 May 2024
TIE: Revolutionizing Text-based Image Editing for Complex-Prompt Following and High-Fidelity Editing Xinyu Zhang Mengxue Kang Fei Wei Shuang Xu Yuhe Liu Lin Ma MLLM DiffM 34 2 0 27 May 2024
SEED-X: Multimodal Models with Unified Multi-granularity Comprehension and Generation Yuying Ge Sijie Zhao Jinguo Zhu Yixiao Ge Kun Yi Lin Song Chen Li Xiaohan Ding Ying Shan VLM 62 107 0 22 Apr 2024
Fact :Teaching MLLMs with Faithful, Concise and Transferable Rationales Minghe Gao Shuang Chen Liang Pang Yuan Yao Jisheng Dang Wenqiao Zhang Juncheng Li Siliang Tang Yueting Zhuang Tat-Seng Chua LRM 37 5 0 17 Apr 2024
Multi-modal Auto-regressive Modeling via Visual Words Tianshuo Peng Zuchao Li Lefei Zhang Hai Zhao Ping Wang Bo Du OffRL 27 4 0 12 Mar 2024
The Revolution of Multimodal Large Language Models: A Survey Davide Caffagni Federico Cocchi Luca Barsellotti Nicholas Moratelli Sara Sarto Lorenzo Baraldi Lorenzo Baraldi Marcella Cornia Rita Cucchiara LRM VLM 56 41 0 19 Feb 2024
Exploring Visual Culture Awareness in GPT-4V: A Comprehensive Probing Yong Cao Wenyan Li Jiaang Li Yifei Yuan Antonia Karamolegkou Daniel Hershcovich VLM 22 7 0 08 Feb 2024
Multimodal Pathway: Improve Transformers with Irrelevant Data from Other Modalities Yiyuan Zhang Xiaohan Ding Kaixiong Gong Yixiao Ge Ying Shan Xiangyu Yue ViT 22 7 0 25 Jan 2024
MM-LLMs: Recent Advances in MultiModal Large Language Models Duzhen Zhang Yahan Yu Jiahua Dong Chenxing Li Dan Su Chenhui Chu Dong Yu OffRL LRM 52 179 0 24 Jan 2024
MM-Interleaved: Interleaved Image-Text Generative Modeling via Multi-modal Feature Synchronizer Changyao Tian Xizhou Zhu Yuwen Xiong Weiyun Wang Zhe Chen ... Tong Lu Jie Zhou Hongsheng Li Yu Qiao Jifeng Dai AuLLM 85 42 0 18 Jan 2024
Masked Modeling for Self-supervised Representation Learning on Vision and Beyond Siyuan Li Luyuan Zhang Zedong Wang Di Wu Lirong Wu ... Jun-Xiong Xia Cheng Tan Yang Liu Baigui Sun Stan Z. Li SSL 36 14 0 31 Dec 2023
MagicBrush: A Manually Annotated Dataset for Instruction-Guided Image Editing Kai Zhang Lingbo Mo Wenhu Chen Huan Sun Yu-Chuan Su EGVM 111 237 0 16 Jun 2023
BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Models Junnan Li Dongxu Li Silvio Savarese Steven C. H. Hoi VLM MLLM 270 4,244 0 30 Jan 2023