Title
RoboSpatial: Teaching Spatial Understanding to 2D and 3D Vision-Language Models for Robotics Chan Hee Song Valts Blukis Jonathan Tremblay Stephen Tyree Yu-Chuan Su Stan Birchfield 249 20 0 25 Nov 2024
Is 'Right' Right? Enhancing Object Orientation Understanding in Multimodal Large Language Models through Egocentric Instruction Tuning Ji Hyeok Jung Eun Tae Kim S. Kim Joo Ho Lee Bumsoo Kim Buru Chang VLM 516 2 0 24 Nov 2024
Benchmarking Multimodal Models for Ukrainian Language Understanding Across Academic and Cultural Domains Yurii Paniv Artur Kiulian Dmytro Chaplynskyi M. Khandoga Anton Polishko Tetiana Bas Guillermo Gabrielli 103 1 0 22 Nov 2024
FuseGPT: Learnable Layers Fusion of Generative Pre-trained Transformers Zehua Pei Hui-Ling Zhen Xianzhi Yu Sinno Jialin Pan Mingxuan Yuan Bei Yu AI4CE 251 3 0 21 Nov 2024
BetterBench: Assessing AI Benchmarks, Uncovering Issues, and Establishing Best Practices Anka Reuel Amelia F. Hardy Chandler Smith Max Lamparth Malcolm Hardy Mykel J. Kochenderfer ELM 199 24 0 20 Nov 2024
Beyond Visual Understanding: Introducing PARROT-360V for Vision Language Model Benchmarking Harsha Vardhan Khurdula Basem Rizk Indus Khaitan Janit Anjaria Aviral Srivastava Rajvardhan Khaitan ELM VLM LRM 99 0 0 20 Nov 2024
AtomThink: A Slow Thinking Framework for Multimodal Mathematical Reasoning Kun Xiang Zhili Liu Zihao Jiang Yunshuang Nie Runhui Huang ... Yihan Zeng Jiawei Han Lanqing Hong Hang Xu Xiaodan Liang LRM 236 17 0 18 Nov 2024
VidComposition: Can MLLMs Analyze Compositions in Compiled Videos? Yunlong Tang Junjia Guo Hang Hua Susan Liang Mingqian Feng ... Chao Huang Jing Bi Zeliang Zhang Pooyan Fazli Chenliang Xu CoGe 159 11 0 17 Nov 2024
Enhancing the Reasoning Ability of Multimodal Large Language Models via Mixed Preference Optimization Weiyun Wang Zhe Chen Wenhai Wang Yue Cao Yangzhou Liu ... Jinguo Zhu X. Zhu Lewei Lu Yu Qiao Jifeng Dai LRM 145 93 1 15 Nov 2024
Multimodal Instruction Tuning with Hybrid State Space Models Jianing Zhou Han Li Shuai Zhang Ning Xie Ruijie Wang Xiaohan Nie Sheng Liu Lingyun Wang 79 0 0 13 Nov 2024
M-Longdoc: A Benchmark For Multimodal Super-Long Document Understanding And A Retrieval-Aware Tuning Framework Yew Ken Chia Liying Cheng Hou Pong Chan Chaoqun Liu Maojia Song Sharifah Mahani Aljunied Soujanya Poria Lidong Bing RALM VLM 112 6 0 09 Nov 2024
HourVideo: 1-Hour Video-Language Understanding Keshigeyan Chandrasegaran Agrim Gupta Lea M. Hadzic Taran Kota Jimming He Cristobal Eyzaguirre Zane Durante Manling Li Jiajun Wu L. Fei-Fei VLM 110 49 0 07 Nov 2024
Needle Threading: Can LLMs Follow Threads through Near-Million-Scale Haystacks? Jonathan Roberts Kai Han Samuel Albanie LLMAG 475 1 0 07 Nov 2024
Medical Adaptation of Large Language and Vision-Language Models: Are We Making Progress? Daniel P. Jeong Saurabh Garg Zachary Chase Lipton Michael Oberst LM&MA VLM ELM 75 14 0 06 Nov 2024
M3SciQA: A Multi-Modal Multi-Document Scientific QA Benchmark for Evaluating Foundation Models Chuhan Li Ziyao Shangguan Yilun Zhao Deyuan Li Yongxu Liu Arman Cohan 80 3 0 06 Nov 2024
Exploring Response Uncertainty in MLLMs: An Empirical Evaluation under Misleading Scenarios Yunkai Dang Mengxi Gao Yibo Yan Xin Zou Yanggan Gu Aiwei Liu Xuming Hu 92 6 0 05 Nov 2024
INQUIRE: A Natural World Text-to-Image Retrieval Benchmark Edward Vendrow Omiros Pantazis Alexander Shepard Gabriel J. Brostow Kate E. Jones Oisin Mac Aodha Sara Beery Grant Van Horn VLM 111 7 0 04 Nov 2024
Attacking Vision-Language Computer Agents via Pop-ups Yanzhe Zhang Tao Yu Diyi Yang AAML VLM 142 34 0 04 Nov 2024
LIBMoE: A Library for comprehensive benchmarking Mixture of Experts in Large Language Models Nam V. Nguyen Thong T. Doan Luong Tran Van Nguyen Quang Pham MoE 175 1 0 01 Nov 2024
Project Sid: Many-agent simulations toward AI civilization Altera. AL Andrew Ahn Nic Becker Stephanie Carroll Nico Christie ... Shuying Luo Peter Y Wang Mathew Willows Feitong Yang Guangyu Robert Yang LLMAG LM&Ro AI4CE 119 15 0 31 Oct 2024
PIP-MM: Pre-Integrating Prompt Information into Visual Encoding via Existing MLLM Structures Tianxiang Wu Minxin Nie Ziqiang Cao MLLM 50 0 0 30 Oct 2024
Vision-Language Models Can Self-Improve Reasoning via Reflection Kanzhi Cheng Yantao Li Fangzhi Xu Jianbing Zhang Hao Zhou Yang Liu ReLM LRM 150 23 0 30 Oct 2024
Towards Unifying Understanding and Generation in the Era of Vision Foundation Models: A Survey from the Autoregression Perspective Shenghao Xie Wenqiang Zu Mingyang Zhao Duo Su Shilong Liu Ruohua Shi Guoqi Li Shanghang Zhang Lei Ma LRM 152 3 0 29 Oct 2024
ProMQA: Question Answering Dataset for Multimodal Procedural Activity Understanding Kimihiro Hasegawa Wiradee Imrattanatrai Zhi-Qi Cheng Masaki Asada Susan Holm Yuran Wang Ken Fukuda Teruko Mitamura 48 1 0 29 Oct 2024
Standardization Trends on Safety and Trustworthiness Technology for Advanced AI Jonghong Jeon 72 2 0 29 Oct 2024
AutoBench-V: Can Large Vision-Language Models Benchmark Themselves? Han Bao Yue Huang Yanbo Wang Jiayi Ye Xiangqi Wang Preslav Nakov Mohamed Elhoseiny Wei Wei Mohamed Elhoseiny Xiangliang Zhang 109 11 0 28 Oct 2024
MatViX: Multimodal Information Extraction from Visually Rich Articles Ghazal Khalighinejad Sharon Scott Ollie Liu Kelly L. Anderson Rickard Stureborg Aman Tyagi Bhuwan Dhingra 55 1 0 27 Oct 2024
MMDocBench: Benchmarking Large Vision-Language Models for Fine-Grained Visual Document Understanding Fengbin Zhu Ziyang Liu Xiang Yao Ng Haohui Wu Wenjie Wang Fuli Feng Chao Wang Huanbo Luan Tat-Seng Chua VLM 102 3 0 25 Oct 2024
COAT: Compressing Optimizer states and Activation for Memory-Efficient FP8 Training Haocheng Xi Han Cai Ligeng Zhu Yaojie Lu Kurt Keutzer Jianfei Chen Song Han MQ 173 11 0 25 Oct 2024
Bongard in Wonderland: Visual Puzzles that Still Make AI Go Mad? Antonia Wüst Tim Nelson Tobiasch Lukas Helff Inga Ibs Wolfgang Stammer Devendra Singh Dhami Constantin Rothkopf Kristian Kersting CoGe ReLM VLM LRM 172 3 0 25 Oct 2024
Revealing and Reducing Gender Biases in Vision and Language Assistants (VLAs) Leander Girrbach Yiran Huang Stephan Alaniz Trevor Darrell Zeynep Akata VLM 145 2 0 25 Oct 2024
MMAU: A Massive Multi-Task Audio Understanding and Reasoning Benchmark S. Sakshi Utkarsh Tyagi Sonal Kumar Ashish Seth Ramaneswaran Selvakumar Oriol Nieto R. Duraiswami Sreyan Ghosh Dinesh Manocha AuLLM ELM 148 46 0 24 Oct 2024
CAMEL-Bench: A Comprehensive Arabic LMM Benchmark Sara Ghaboura Ahmed Heakl Omkar Thawakar Ali Alharthi Ines Riahi Abduljalil Saif Jorma T. Laaksonen Fahad Shahbaz Khan Salman Khan Rao Muhammad Anwer 84 3 0 24 Oct 2024
Infinity-MM: Scaling Multimodal Performance with Large-Scale and High-Quality Instruction Data Shuhao Gu Jialing Zhang Siyuan Zhou Kevin Yu Zhaohu Xing ... Yufeng Cui Xinlong Wang Yaoqi Liu Fangxiang Feng Guang Liu SyDa VLM MLLM 124 29 0 24 Oct 2024
TP-Eval: Tap Multimodal LLMs' Potential in Evaluation by Customizing Prompts Yuxuan Xie Tianhua Li Wenqi Shao Kai Zhang 66 0 0 23 Oct 2024
Benchmarking Foundation Models on Exceptional Cases: Dataset Creation and Validation Suho Kang Jungyang Park Joonseo Ha SoMin Kim JinHyeong Kim Subeen Park Kyungwoo Song LRM 58 0 0 23 Oct 2024
ADEM-VL: Adaptive and Embedded Fusion for Efficient Vision-Language Tuning Zhiwei Hao Jianyuan Guo Li Shen Yong Luo Han Hu Yonggang Wen VLM 97 0 0 23 Oct 2024
MIA-DPO: Multi-Image Augmented Direct Preference Optimization For Large Vision-Language Models Ziyu Liu Yuhang Zang Xiaoyi Dong Pan Zhang Yuhang Cao Haodong Duan Zeang Sheng Yuanjun Xiong Dahua Lin Jiaqi Wang 108 12 0 23 Oct 2024
Meaning Typed Prompting: A Technique for Efficient, Reliable Structured Output Generation Chandra Irugalbandara 61 0 0 22 Oct 2024
Do Vision-Language Models Represent Space and How? Evaluating Spatial Frame of Reference Under Ambiguities Zheyuan Zhang Fengyuan Hu Jayjun Lee Freda Shi Parisa Kordjamshidi Joyce Chai Ziqiao Ma 157 15 0 22 Oct 2024
JMMMU: A Japanese Massive Multi-discipline Multimodal Understanding Benchmark for Culture-aware Evaluation Shota Onohara Atsuyuki Miyai Yuki Imajuku Kazuki Egashira Jeonghun Baek Xiang Yue Graham Neubig Kiyoharu Aizawa OSLM 260 6 0 22 Oct 2024
Teach Multimodal LLMs to Comprehend Electrocardiographic Images Ruoqi Liu Yuelin Bai Xiang Yue Ping Zhang 56 5 0 21 Oct 2024
Mini-InternVL: A Flexible-Transfer Pocket Multimodal Model with 5% Parameters and 90% Performance Zhangwei Gao Zhe Chen Erfei Cui Yiming Ren Weiyun Wang ... Lewei Lu Tong Lu Yu Qiao Jifeng Dai Wenhai Wang VLM 169 40 0 21 Oct 2024
Griffon-G: Bridging Vision-Language and Vision-Centric Tasks via Large Multimodal Models Yufei Zhan Hongyin Zhao Yousong Zhu Fan Yang Ming Tang Jinqiao Wang MLLM 90 1 0 21 Oct 2024
OpenMU: Your Swiss Army Knife for Music Understanding Mengjie Zhao Zhi-Wei Zhong Zhuoyuan Mao Shiqi Yang Wei-Hsiang Liao Shusuke Takahashi Hiromi Wakaki Yuki Mitsufuji OSLM 103 8 0 21 Oct 2024
LLaVA-KD: A Framework of Distilling Multimodal Large Language Models Y. Cai Jiangning Zhang Haoyang He Xinwei He Ao Tong Zhenye Gan Chengjie Wang Zhucun Xue Yong-Jin Liu X. Bai VLM 96 6 0 21 Oct 2024
Dynamic Intelligence Assessment: Benchmarking LLMs on the Road to AGI with a Focus on Model Confidence Norbert Tihanyi Tamás Bisztray Richard A. Dubniczky Rebeka Tóth B. Borsos ... Ryan Marinelli Lucas C. Cordeiro Merouane Debbah Vasileios Mavroeidis Audun Josang 95 5 0 20 Oct 2024
NaturalBench: Evaluating Vision-Language Models on Natural Adversarial Samples Baiqi Li Zhiqiu Lin Wenxuan Peng Jean de Dieu Nyandwi Daniel Jiang Zixian Ma Simran Khanuja Ranjay Krishna Graham Neubig Deva Ramanan AAML CoGe VLM 234 31 0 18 Oct 2024
LabSafety Bench: Benchmarking LLMs on Safety Issues in Scientific Labs Yujun Zhou Jingdong Yang Yue Huang Kehan Guo Zoe Emory ... Tian Gao Werner Geyer Nuno Moniz Nitesh Chawla Xiangliang Zhang 134 7 0 18 Oct 2024
Efficient Vision-Language Models by Summarizing Visual Tokens into Compact Registers Yuxin Wen Qingqing Cao Qichen Fu Sachin Mehta Mahyar Najibi VLM 127 5 0 17 Oct 2024