Vary: Scaling up the Vision Vocabulary for Large Vision-Language Models

11 December 2023

Haoran Wei

Lingyu Kong

Jinyue Chen

Liang Zhao

Zheng Ge

Jinrong Yang

Jian‐Yuan Sun

Chunrui Han

Xiangyu Zhang

MLLM

VLM

ArXiv (abs)PDF HTML

Papers citing "Vary: Scaling up the Vision Vocabulary for Large Vision-Language Models"

22 / 22 papers shown

Title
GDI-Bench: A Benchmark for General Document Intelligence with Vision and Reasoning Decoupling Siqi Li Yufan Shen Xiangnan Chen Jiayi Chen Hengwei Ju ... Botian Shi Y. Liu Xinyu Cai Yu Qiao Yu Qiao VLM ELM 165 1 0 30 Apr 2025
Skip-Vision: Efficient and Scalable Acceleration of Vision-Language Models via Adaptive Token Skipping Weili Zeng Ziyuan Huang Kaixiang Ji Yichao Yan VLM 218 1 0 26 Mar 2025
Enhancing Complex Formula Recognition with Hierarchical Detail-Focused Network Jiale Wang Junhui Yu Huanyong Liu Chenanran Kong AIMat 85 0 0 10 Jan 2025
Law of Vision Representation in MLLMs Shijia Yang Bohan Zhai Quanzeng You Jianbo Yuan Hongxia Yang Chenfeng Xu 91 12 0 29 Aug 2024
TextSquare: Scaling up Text-Centric Visual Instruction Tuning Jingqun Tang Chunhui Lin Zhen Zhao Shubo Wei Binghong Wu ... Yuliang Liu Xiang Bai Can Huang Xiang Bai Can Huang LRM VLM MLLM 150 30 0 19 Apr 2024
Qwen Technical Report Jinze Bai Shuai Bai Yunfei Chu Zeyu Cui Kai Dang ... Zhenru Zhang Chang Zhou Jingren Zhou Xiaohuan Zhou Tianhang Zhu OSLM 264 1,895 0 28 Sep 2023
ChatSpot: Bootstrapping Multimodal LLMs via Precise Referring Instruction Tuning Liang Zhao En Yu Zheng Ge Jinrong Yang Hao-Ran Wei ... Jian‐Yuan Sun Yuang Peng Runpei Dong Chunrui Han Xiangyu Zhang MLLM LRM 62 54 0 18 Jul 2023
HuggingGPT: Solving AI Tasks with ChatGPT and its Friends in Hugging Face Yongliang Shen Kaitao Song Xu Tan Dongsheng Li Weiming Lu Yueting Zhuang MLLM 122 908 0 30 Mar 2023
TaskMatrix.AI: Completing Tasks by Connecting Foundation Models with Millions of APIs Yaobo Liang Chenfei Wu Ting Song Wenshan Wu Yan Xia ... Shaoguang Mao Yuntao Wang Linjun Shou Ming Gong Nan Duan LLMAG CLL 77 205 0 29 Mar 2023
MM-REACT: Prompting ChatGPT for Multimodal Reasoning and Action Zhengyuan Yang Linjie Li Jianfeng Wang Kevin Qinghong Lin E. Azarnasab Faisal Ahmed Zicheng Liu Ce Liu Michael Zeng Lijuan Wang ReLM KELM LRM 105 394 0 20 Mar 2023
GLM-130B: An Open Bilingual Pre-trained Model Aohan Zeng Xiao Liu Zhengxiao Du Zihan Wang Hanyu Lai ... Jidong Zhai Wenguang Chen Peng Zhang Yuxiao Dong Jie Tang BDL LRM 355 1,094 0 05 Oct 2022
Emergent Abilities of Large Language Models Jason W. Wei Yi Tay Rishi Bommasani Colin Raffel Barret Zoph ... Tatsunori Hashimoto Oriol Vinyals Percy Liang J. Dean W. Fedus ELM ReLM LRM 286 2,511 0 15 Jun 2022
Flamingo: a Visual Language Model for Few-Shot Learning Jean-Baptiste Alayrac Jeff Donahue Pauline Luc Antoine Miech Iain Barr ... Mikolaj Binkowski Ricardo Barreira Oriol Vinyals Andrew Zisserman Karen Simonyan MLLM VLM 418 3,602 0 29 Apr 2022
End-to-end Document Recognition and Understanding with Dessurt Brian L. Davis B. Morse Brian L. Price Chris Tensmeyer Curtis Wigington Vlad I. Morariu VLM ViT 90 73 0 30 Mar 2022
ChartQA: A Benchmark for Question Answering about Charts with Visual and Logical Reasoning Ahmed Masry Do Xuan Long J. Tan Shafiq Joty Enamul Hoque AIMat 134 684 0 19 Mar 2022
LaTr: Layout-Aware Transformer for Scene-Text VQA Ali Furkan Biten Ron Litman Yusheng Xie Srikar Appalaraju R. Manmatha ViT 95 102 0 23 Dec 2021
LAION-400M: Open Dataset of CLIP-Filtered 400 Million Image-Text Pairs Christoph Schuhmann Richard Vencu Romain Beaumont R. Kaczmarczyk Clayton Mullis Aarush Katta Theo Coombes J. Jitsev Aran Komatsuzaki VLM MLLM CLIP 243 1,441 0 03 Nov 2021
DocVQA: A Dataset for VQA on Document Images Minesh Mathew Dimosthenis Karatzas C. V. Jawahar 144 743 0 01 Jul 2020
Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer Colin Raffel Noam M. Shazeer Adam Roberts Katherine Lee Sharan Narang Michael Matena Yanqi Zhou Wei Li Peter J. Liu AIMat 462 20,317 0 23 Oct 2019
Towards VQA Models That Can Read Amanpreet Singh Vivek Natarajan Meet Shah Yu Jiang Xinlei Chen Dhruv Batra Devi Parikh Marcus Rohrbach EgoV 111 1,253 0 18 Apr 2019
SGDR: Stochastic Gradient Descent with Warm Restarts I. Loshchilov Frank Hutter ODL 350 8,174 0 13 Aug 2016
Microsoft COCO: Common Objects in Context Nayeon Lee Michael Maire Serge J. Belongie Lubomir Bourdev Ross B. Girshick James Hays Pietro Perona Deva Ramanan C. L. Zitnick Piotr Dollár ObjD 424 43,814 0 01 May 2014