Feast Your Eyes: Mixture-of-Resolution Adaptation for Multimodal Large Language Models

5 March 2024

Papers citing "Feast Your Eyes: Mixture-of-Resolution Adaptation for Multimodal Large Language Models"

46 / 46 papers shown

Title
Visual Instruction Tuning with Chain of Region-of-Interest Yixin Chen Shuai Zhang Boran Han Bernie Wang 26 0 0 11 May 2025
Zoomer: Adaptive Image Focus Optimization for Black-box MLLM Jiaxu Qian Chendong Wang Yi Yang Chaoyun Zhang Huiqiang Jiang ... Saravan Rajmohan Dongmei Zhang Xuming Hu Qi Zhang Lili Qiu VLM 84 0 0 30 Apr 2025
ActionArt: Advancing Multimodal Large Models for Fine-Grained Human-Centric Video Understanding Yi-Xing Peng Q. Yang Yu-Ming Tang Shenghao Fu Kun-Yu Lin Xihan Wei Wei-Shi Zheng 45 0 0 25 Apr 2025
HRScene: How Far Are VLMs from Effective High-Resolution Image Understanding? Yusen Zhang Wenliang Zheng Aashrith Madasu Peng Shi Ryo Kamoi ... Ranran Haoran Zhang Avitej Iyer Renze Lou Wenpeng Yin Rui Zhang 68 0 0 25 Apr 2025
LEO-MINI: An Efficient Multimodal Large Language Model using Conditional Token Reduction and Mixture of Multi-Modal Experts Yimu Wang Mozhgan Nasr Azadani Sean Sedwards Krzysztof Czarnecki MLLM MoE 52 0 0 07 Apr 2025
Dynamic Pyramid Network for Efficient Multimodal Large Language Model Hao Ai Kunyi Wang Zezhou Wang H. Lu Jin Tian Yaxin Luo Peng-Fei Xing Jen-Yuan Huang Huaxia Li Gen Luo MLLM VLM 110 0 0 26 Mar 2025
MLLM-Selector: Necessity and Diversity-driven High-Value Data Selection for Enhanced Visual Instruction Tuning Yiwei Ma Guohai Xu Xiaoshuai Sun Jiayi Ji Jie Lou Debing Zhang Rongrong Ji 95 0 0 26 Mar 2025
Scaling Vision Pre-Training to 4K Resolution Baifeng Shi Boyi Li Han Cai Yaojie Lu Sifei Liu ... Jan Kautz Enze Xie Trevor Darrell Pavlo Molchanov Hongxu Yin CLIP 139 0 0 25 Mar 2025
When Large Vision-Language Model Meets Large Remote Sensing Imagery: Coarse-to-Fine Text-Guided Token Pruning Junwei Luo Yingying Zhang Xiaoyu Yang Kang Wu Qi Zhu Lei Liang Jingdong Chen Yansheng Li 67 1 0 10 Mar 2025
Keeping Yourself is Important in Downstream Tuning Multimodal Large Language Model Wenke Huang Jian Liang Xianda Guo Yiyang Fang Guancheng Wan ... Bin Yang He Li Jiawei Shao Mang Ye Bo Du OffRL LRM MLLM KELM VLM 65 1 0 06 Mar 2025
MOVE: A Mixture-of-Vision-Encoders Approach for Domain-Focused Vision-Language Processing Matvey Skripkin Elizaveta Goncharova Dmitrii Tarasov Andrey Kuznetsov 67 0 0 24 Feb 2025
MLLMs Know Where to Look: Training-free Perception of Small Visual Details with Multimodal LLMs Jiarui Zhang Mahyar Khayatkhoei P. Chhikara Filip Ilievski LRM 39 6 0 24 Feb 2025
LEO: Boosting Mixture of Vision Encoders for Multimodal Large Language Models Mozhgan Nasr Azadani James Riddell Sean Sedwards Krzysztof Czarnecki MLLM VLM 47 2 0 13 Jan 2025
LLaVA-UHD v2: an MLLM Integrating High-Resolution Semantic Pyramid via Hierarchical Window Transformer Yipeng Zhang Yijia Liu Zonghao Guo Yidan Zhang Xuesong Yang ... Yuan Yao Zhiyuan Liu Tat-Seng Chua Maosong Sun Maosong Sun MLLM VLM 86 0 0 18 Dec 2024
SoftVQ-VAE: Efficient 1-Dimensional Continuous Tokenizer Hongyu Chen Z. Wang Xianrui Li Xingwu Sun Fangyi Chen Jiang Liu Jiadong Wang Bhiksha Raj Zicheng Liu Emad Barsoum VLM 114 6 0 14 Dec 2024
Beyond Text-Visual Attention: Exploiting Visual Cues for Effective Token Pruning in VLMs Qizhe Zhang Aosong Cheng Ming Lu Zhiyong Zhuo Minqi Wang Jiajun Cao Shaobo Guo Qi She Shanghang Zhang VLM 90 11 0 02 Dec 2024
ChatRex: Taming Multimodal LLM for Joint Perception and Understanding Qing Jiang Gen Luo Yuqin Yang Yuda Xiong Yihao Chen Zhaoyang Zeng Tianhe Ren Lei Zhang VLM LRM 109 6 0 27 Nov 2024
ZoomEye: Enhancing Multimodal LLMs with Human-Like Zooming Capabilities through Tree-Based Image Exploration Haozhan Shen Kangjia Zhao Tiancheng Zhao Ruochen Xu Zilun Zhang Mingwei Zhu Jianwei Yin 97 4 0 25 Nov 2024
FINECAPTION: Compositional Image Captioning Focusing on Wherever You Want at Any Granularity Hang Hua Qing Liu Lingzhi Zhang Jing Shi Zhifei Zhang Yilin Wang Jianming Zhang Jiebo Luo CoGe VLM 95 6 0 23 Nov 2024
FocusLLaVA: A Coarse-to-Fine Approach for Efficient and Effective Visual Token Compression Yuke Zhu Chi Xie Shuang Liang Bo Zheng Sheng Guo 80 8 0 21 Nov 2024
Multimodal Instruction Tuning with Hybrid State Space Models Jianing Zhou Han Li Shuai Zhang Ning Xie Ruijie Wang Xiaohan Nie Sheng Liu Lingyun Wang 40 0 0 13 Nov 2024
SV-RAG: LoRA-Contextualizing Adaptation of MLLMs for Long Document Understanding Jian Chen R. Zhang Yufan Zhou Tong Yu Franck Dernoncourt J. Gu Ryan A. Rossi Changyou Chen Tong Sun 39 0 0 02 Nov 2024
Mini-InternVL: A Flexible-Transfer Pocket Multimodal Model with 5% Parameters and 90% Performance Zhangwei Gao Zhe Chen Erfei Cui Yiming Ren Weiyun Wang ... Lewei Lu Tong Lu Yu Qiao Jifeng Dai Wenhai Wang VLM 70 24 0 21 Oct 2024
LLaVA-KD: A Framework of Distilling Multimodal Large Language Models Y. Cai Jiangning Zhang Haoyang He Xinwei He Ao Tong Zhenye Gan Chengjie Wang X. Bai VLM 29 2 0 21 Oct 2024
$γ-$ MoD: Exploring Mixture-of-Depth Adaptation for Multimodal Large Language Models Yaxin Luo Gen Luo Jiayi Ji Yiyi Zhou Xiaoshuai Sun Zhiqiang Shen Rongrong Ji VLM MoE 45 1 0 17 Oct 2024
Improving Multi-modal Large Language Model through Boosting Vision Capabilities Yanpeng Sun H. Zhang Qiang Chen Xinyu Zhang Nong Sang Gang Zhang Jingdong Wang Zechao Li 31 5 0 17 Oct 2024
MMFuser: Multimodal Multi-Layer Feature Fuser for Fine-Grained Vision-Language Understanding Yue Cao Yangzhou Liu Zhe Chen Guangchen Shi Wenhai Wang Danhuai Zhao Tong Lu 51 5 0 15 Oct 2024
MMCOMPOSITION: Revisiting the Compositionality of Pre-trained Vision-Language Models Hang Hua Yunlong Tang Ziyun Zeng Liangliang Cao Zhengyuan Yang Hangfeng He Chenliang Xu Jiebo Luo VLM CoGe 44 9 0 13 Oct 2024
EMOVA: Empowering Language Models to See, Hear and Speak with Vivid Emotions Kai Chen Yunhao Gou Runhui Huang Zhili Liu Daxin Tan ... Qun Liu Jun Yao Lu Hou Hang Xu Hang Xu AuLLM MLLM VLM 82 21 0 26 Sep 2024
Eagle: Exploring The Design Space for Multimodal LLMs with Mixture of Encoders Min Shi Fuxiao Liu Shihao Wang Shijia Liao Subhashree Radhakrishnan ... Andrew Tao Andrew Tao Zhiding Yu Guilin Liu Guilin Liu MLLM 30 53 0 28 Aug 2024
ControlMLLM: Training-Free Visual Prompt Learning for Multimodal Large Language Models Ming-Kuan Wu Xinyue Cai Jiayi Ji Jiale Li Oucheng Huang Gen Luo Hao Fei Xiaoshuai Sun Rongrong Ji MLLM 51 7 0 31 Jul 2024
Advancing Multimodal Large Language Models in Chart Question Answering with Visualization-Referenced Instruction Tuning Xingchen Zeng Haichuan Lin Yilin Ye Wei Zeng 57 15 0 29 Jul 2024
LLaVA-Read: Enhancing Reading Ability of Multimodal Language Models Ruiyi Zhang Yufan Zhou Jian Chen Jiuxiang Gu Changyou Chen Tongfei Sun VLM 41 6 0 27 Jul 2024
INF-LLaVA: Dual-perspective Perception for High-Resolution Multimodal Large Language Model Yiwei Ma Zhibin Wang Xiaoshuai Sun Weihuang Lin Qiang-feng Zhou Jiayi Ji Rongrong Ji MLLM VLM 57 1 0 23 Jul 2024
HiRes-LLaVA: Restoring Fragmentation Input in High-Resolution Large Vision-Language Models Runhui Huang Xinpeng Ding Chunwei Wang J. N. Han Yulong Liu Hengshuang Zhao Hang Xu Lu Hou Wei Zhang Xiaodan Liang VLM 31 8 0 11 Jul 2024
MG-LLaVA: Towards Multi-Granularity Visual Instruction Tuning Xiangyu Zhao Xiangtai Li Haodong Duan Haian Huang Yining Li Kai Chen Hua Yang VLM MLLM 45 10 0 25 Jun 2024
SkySenseGPT: A Fine-Grained Instruction Tuning Dataset and Model for Remote Sensing Vision-Language Understanding Junwei Luo Zhen Pang Yongjun Zhang Tingzhu Wang Linlin Wang ... Jiangwei Lao Jian Wang Jingdong Chen Yihua Tan Yansheng Li 48 21 0 14 Jun 2024
GPT-4o: Visual perception performance of multimodal large language models in piglet activity understanding Yiqi Wu Xiaodan Hu Ziming Fu Siling Zhou Jiangong Li MLLM 35 10 0 14 Jun 2024
Parameter-Inverted Image Pyramid Networks Xizhou Zhu Xue Yang Zhaokai Wang Hao Li Wenhan Dou Junqi Ge Lewei Lu Yu Qiao Jifeng Dai 47 0 0 06 Jun 2024
ConvLLaVA: Hierarchical Backbones as Visual Encoder for Large Multimodal Models Chunjiang Ge Sijie Cheng Ziming Wang Jiale Yuan Yuan Gao Jun Song Shiji Song Gao Huang Bo Zheng MLLM VLM 36 17 0 24 May 2024
Meteor: Mamba-based Traversal of Rationale for Large Language and Vision Models Byung-Kwan Lee Chae Won Kim Beomchan Park Yonghyun Ro MLLM LRM 41 18 0 24 May 2024
CityLLaVA: Efficient Fine-Tuning for VLMs in City Scenario Zhizhao Duan Hao Cheng Duo Xu Xi Wu Xiangxie Zhang Xi Ye Zhen Xie 29 7 0 06 May 2024
How Far Are We to GPT-4V? Closing the Gap to Commercial Multimodal Models with Open-Source Suites Zhe Chen Weiyun Wang Hao Tian Shenglong Ye Zhangwei Gao ... Tong Lu Dahua Lin Yu Qiao Jifeng Dai Wenhai Wang MLLM VLM 49 533 0 25 Apr 2024
LLaVA-UHD: an LMM Perceiving Any Aspect Ratio and High-Resolution Images Ruyi Xu Yuan Yao Zonghao Guo Junbo Cui Zanlin Ni Chunjiang Ge Tat-Seng Chua Zhiyuan Liu Maosong Sun Gao Huang VLM MLLM 37 103 0 18 Mar 2024
Visual Chain of Thought: Bridging Logical Gaps with Multimodal Infillings Daniel Philip Rose Vaishnavi Himakunthala Andy Ouyang Ryan He Alex Mei Yujie Lu Michael Stephen Saxon Chinmay Sonar Diba Mirza William Yang Wang LRM 70 37 0 03 May 2023
Learn to Explain: Multimodal Reasoning via Thought Chains for Science Question Answering Pan Lu Swaroop Mishra Tony Xia Liang Qiu Kai-Wei Chang Song-Chun Zhu Oyvind Tafjord Peter Clark A. Kalyan ELM ReLM LRM 211 1,106 0 20 Sep 2022