TRINS: Towards Multimodal Language Models that Can Read

TRINS: Towards Multimodal Language Models that Can Read

10 June 2024

Jiuxiang Gu

Changyou Chen

Tong Sun

Papers citing "TRINS: Towards Multimodal Language Models that Can Read"

13 / 13 papers shown

Title
Towards Visual Text Grounding of Multimodal Large Language Model Ming Li Ruiyi Zhang Jian Chen Jiuxiang Gu Yufan Zhou Franck Dernoncourt Wanrong Zhu Dinesh Manocha Tong Sun 41 2 0 07 Apr 2025
A High-Quality Text-Rich Image Instruction Tuning Dataset via Hybrid Instruction Generation Shijie Zhou R. Zhang Yufan Zhou Changyou Chen VLM 77 1 0 20 Dec 2024
TextLap: Customizing Language Models for Text-to-Layout Planning Jian Chen Ruiyi Zhang Yufan Zhou Jennifer Healey J. Gu Zhiqiang Xu Chong Chen VLM 44 3 0 09 Oct 2024
LLaVA-Read: Enhancing Reading Ability of Multimodal Language Models Ruiyi Zhang Yufan Zhou Jian Chen Jiuxiang Gu Changyou Chen Tongfei Sun VLM 41 6 0 27 Jul 2024
mPLUG-Owl2: Revolutionizing Multi-modal Large Language Model with Modality Collaboration Qinghao Ye Haiyang Xu Jiabo Ye Mingshi Yan Anwen Hu Haowei Liu Qi Qian Ji Zhang Fei Huang Jingren Zhou MLLM VLM 126 379 0 07 Nov 2023
MiniGPT-v2: large language model as a unified interface for vision-language multi-task learning Jun Chen Deyao Zhu Xiaoqian Shen Xiang Li Zechun Liu Pengchuan Zhang Raghuraman Krishnamoorthi Vikas Chandra Yunyang Xiong Mohamed Elhoseiny MLLM 160 443 0 14 Oct 2023
UReader: Universal OCR-free Visually-situated Language Understanding with Multimodal Large Language Model Jiabo Ye Anwen Hu Haiyang Xu Qinghao Ye Mingshi Yan ... Ji Zhang Qin Jin Liang He Xin Lin Feiyan Huang VLM MLLM 126 85 0 08 Oct 2023
Kosmos-2.5: A Multimodal Literate Model Tengchao Lv Yupan Huang Jingye Chen Lei Cui Shuming Ma ... Weiyao Luo Shaoxiang Wu Guoxin Wang Cha Zhang Furu Wei VLM MLLM 34 63 0 20 Sep 2023
Multimodal Foundation Models: From Specialists to General-Purpose Assistants Chunyuan Li Zhe Gan Zhengyuan Yang Jianwei Yang Linjie Li Lijuan Wang Jianfeng Gao MLLM 118 228 0 18 Sep 2023
Tag2Text: Guiding Vision-Language Model via Image Tagging Xinyu Huang Youcai Zhang Jinyu Ma Weiwei Tian Rui Feng Yuejie Zhang Yaqian Li Yandong Guo Lei Zhang CLIP MLLM VLM 3DV 66 74 0 10 Mar 2023
BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Models Junnan Li Dongxu Li Silvio Savarese Steven C. H. Hoi VLM MLLM 287 4,261 0 30 Jan 2023
Training language models to follow instructions with human feedback Long Ouyang Jeff Wu Xu Jiang Diogo Almeida Carroll L. Wainwright ... Amanda Askell Peter Welinder Paul Christiano Jan Leike Ryan J. Lowe OSLM ALM 339 12,003 0 04 Mar 2022
Conceptual 12M: Pushing Web-Scale Image-Text Pre-Training To Recognize Long-Tail Visual Concepts Soravit Changpinyo P. Sharma Nan Ding Radu Soricut VLM 296 1,084 0 17 Feb 2021