Title
A Comprehensive Survey of Large Language Models and Multimodal Large Language Models in Medicine Hanguang Xiao Feizhong Zhou Xianglong Liu Tianqi Liu Zhipeng Li Xin Liu Xiaoxuan Huang AILaw LM&MA LRM 91 25 0 31 Dec 2024
A Simple Recipe for Contrastively Pre-training Video-First Encoders Beyond 16 Frames Pinelopi Papalampidi Skanda Koppula Shreya Pathak Justin T Chiu Joseph Heyward Viorica Patraucean Jiajun Shen Antoine Miech Andrew Zisserman Aida Nematzdeh VLM 98 26 0 31 Dec 2024
CAD-Assistant: Tool-Augmented VLLMs as Generic CAD Task Solvers Dimitrios Mallis Ahmet Serdar Karadeniz Sebastian Cavada Danila Rukhovich Niki Maria Foteinopoulou K. Cherenkova Anis Kacem Djamila Aouada 130 6 0 18 Dec 2024
GeoX: Geometric Problem Solving Through Unified Formalized Vision-Language Pre-training Renqiu Xia Mingxing Li Hancheng Ye Wenjie Wu Hongbin Zhou ... Zeang Sheng Botian Shi Tao Chen Junchi Yan Bo Zhang 126 10 0 16 Dec 2024
Do large language vision models understand 3D shapes? Sagi Eppel 3DV 162 1 0 14 Dec 2024
Neptune: The Long Orbit to Benchmarking Long Video Understanding Arsha Nagrani Ruotong Wang Ramin Mehran Rachel Hornung N. B. Gundavarapu ... Boqing Gong Cordelia Schmid Mikhail Sirotenko Yukun Zhu Tobias Weyand 139 8 0 12 Dec 2024
Olympus: A Universal Task Router for Computer Vision Tasks Yuanze Lin Yunsheng Li Dongdong Chen Weijian Xu Ronald Clark Philip Torr VLM ObjD 427 0 0 12 Dec 2024
EgoPlan-Bench2: A Benchmark for Multimodal Large Language Model Planning in Real-World Scenarios Lu Qiu Yuying Ge Yi Chen Yixiao Ge Ying Shan Xihui Liu LLMAG LRM 151 6 0 05 Dec 2024
AdvDreamer Unveils: Are Vision-Language Models Truly Ready for Real-World 3D Variations? Shouwei Ruan Hanqin Liu Yao Huang Xiaoqi Wang Caixin Kang Hang Su Yinpeng Dong Xingxing Wei VGen 155 0 0 04 Dec 2024
COSMOS: Cross-Modality Self-Distillation for Vision Language Pre-training Sanghwan Kim Rui Xiao Mariana-Iuliana Georgescu Stephan Alaniz Zeynep Akata VLM 245 3 0 02 Dec 2024
SeqAfford: Sequential 3D Affordance Reasoning via Multimodal Large Language Model Chunlin Yu Hanqing Wang Ye Shi Haoyang Luo Sibei Yang Jingyi Yu Jingya Wang LRM LM&Ro 148 2 0 02 Dec 2024
On Domain-Specific Post-Training for Multimodal Large Language Models Daixuan Cheng Shaohan Huang Ziyu Zhu Xintong Zhang Wayne Xin Zhao Zhongzhi Luan Bo Dai Zhenliang Zhang VLM 135 4 0 29 Nov 2024
Critic-V: VLM Critics Help Catch VLM Errors in Multimodal Reasoning Di Zhang Jingdi Lei Junxian Li Xunzhi Wang Yong Liu ... Steve Yang Jianbo Wu Peng Ye Wanli Ouyang Dongzhan Zhou OffRL LRM 133 8 0 27 Nov 2024
ChatRex: Taming Multimodal LLM for Joint Perception and Understanding Qing Jiang Gen Luo Yuqin Yang Yuda Xiong Yihao Chen Zhaoyang Zeng Tianhe Ren Lei Zhang VLM LRM 143 8 0 27 Nov 2024
TechCoach: Towards Technical-Point-Aware Descriptive Action Coaching Yuan-Ming Li An-Lan Wang Kun-Yu Lin Yu-Ming Tang Ling-an Zeng Jian-Fang Hu Wei-Shi Zheng 151 6 0 26 Nov 2024
GEMeX: A Large-Scale, Groundable, and Explainable Medical VQA Benchmark for Chest X-ray Diagnosis Bo Liu K. Zou Liming Zhan Zexin Lu Xiaoyu Dong Yidi Chen Chengqiang Xie Jiannong Cao Xiao-Ming Wu Huazhu Fu 169 2 0 25 Nov 2024
SAR3D: Autoregressive 3D Object Generation and Understanding via Multi-scale 3D VQVAE Yongwei Chen Yushi Lan Shangchen Zhou Tengfei Wang Xingang Pan 165 6 0 25 Nov 2024
Is 'Right' Right? Enhancing Object Orientation Understanding in Multimodal Large Language Models through Egocentric Instruction Tuning Ji Hyeok Jung Eun Tae Kim S. Kim Joo Ho Lee Bumsoo Kim Buru Chang VLM 413 1 0 24 Nov 2024
Teaching VLMs to Localize Specific Objects from In-context Examples Sivan Doveh Nimrod Shabtay Wei Lin Eli Schwartz Hilde Kuehne ... Leonid Karlinsky James Glass Assaf Arbelle S. Ullman Muhammad Jehanzeb Mirza VLM 143 1 0 20 Nov 2024
VILA-M3: Enhancing Vision-Language Models with Medical Expert Knowledge Vishwesh Nath Wenqi Li Dong Yang Andriy Myronenko Mingxin Zheng ... Holger Roth Daguang Xu Baris Turkbey Holger Roth Daguang Xu VLM 153 6 0 19 Nov 2024
ITACLIP: Boosting Training-Free Semantic Segmentation with Image, Text, and Architectural Enhancements M. Arda Aydın Efe Mert Çırpar Elvin Abdinli Gözde B. Ünal Y. Sahin VLM 217 1 0 18 Nov 2024
Enhancing the Reasoning Ability of Multimodal Large Language Models via Mixed Preference Optimization Weiyun Wang Zhe Chen Wenhai Wang Yue Cao Yangzhou Liu ... Jinguo Zhu X. Zhu Lewei Lu Yu Qiao Jifeng Dai LRM 102 74 1 15 Nov 2024
Spider: Any-to-Many Multimodal LLM Jinxiang Lai Jie Zhang Jun Liu Jian Li Xiaocheng Lu Song Guo MLLM 118 2 0 14 Nov 2024
VideoGLaMM: A Large Multimodal Model for Pixel-Level Visual Grounding in Videos Shehan Munasinghe Hanan Gani Wenqi Zhu Jiale Cao Eric P. Xing Fahad Shahbaz Khan Salman Khan MLLM VGen VLM 70 6 0 07 Nov 2024
One VLM to Keep it Learning: Generation and Balancing for Data-free Continual Visual Question Answering Deepayan Das Davide Talon Massimiliano Mancini Yiming Wang Elisa Ricci 103 0 0 04 Nov 2024
On Learning Multi-Modal Forgery Representation for Diffusion Generated Video Detection Xiufeng Song Xiao Guo Junxuan Zhang Qirui Li Lei Bai Xiaoming Liu Guangtao Zhai Xiaohong Liu DiffM VGen 114 9 0 31 Oct 2024
R-LLaVA: Improving Med-VQA Understanding through Visual Region of Interest Xupeng Chen Zhixin Lai Kangrui Ruan Shichu Chen Jiaxiang Liu Zuozhu Liu 90 1 0 27 Oct 2024
Revealing and Reducing Gender Biases in Vision and Language Assistants (VLAs) Leander Girrbach Yiran Huang Stephan Alaniz Trevor Darrell Zeynep Akata VLM 92 2 0 25 Oct 2024
Parameter-Efficient Fine-Tuning in Large Models: A Survey of Methodologies Liwen Wang Sheng Chen Linnan Jiang Shu Pan Runze Cai Sen Yang Fei Yang 114 5 0 24 Oct 2024
Do Vision-Language Models Represent Space and How? Evaluating Spatial Frame of Reference Under Ambiguities Zheyuan Zhang Fengyuan Hu Jayjun Lee Freda Shi Parisa Kordjamshidi Joyce Chai Ziqiao Ma 114 13 0 22 Oct 2024
LatentBKI: Open-Dictionary Continuous Mapping in Visual-Language Latent Spaces with Quantifiable Uncertainty Joey Wilson Ruihan Xu Yile Sun Parker Ewen Minghan Zhu Kira Barton Maani Ghaffari 68 0 0 15 Oct 2024
Transforming Game Play: A Comparative Study of DCQN and DTQN Architectures in Reinforcement Learning William A. Stigall 90 0 0 14 Oct 2024
Locality Alignment Improves Vision-Language Models Ian Covert Tony Sun James Zou Tatsunori Hashimoto VLM 189 6 0 14 Oct 2024
VisRAG: Vision-based Retrieval-augmented Generation on Multi-modality Documents S. Yu C. Tang Bokai Xu Junbo Cui Junhao Ran ... Zhenghao Liu Shuo Wang Xu Han Zhiyuan Liu Maosong Sun VLM 142 32 0 14 Oct 2024
3DArticCyclists: Generating Synthetic Articulated 8D Pose-Controllable Cyclist Data for Computer Vision Applications Eduardo R. Corral-Soto Yang Liu Tongtong Cao Y. Ren Liu Bingbing 85 5 0 14 Oct 2024
MRAG-Bench: Vision-Centric Evaluation for Retrieval-Augmented Multimodal Models Wenbo Hu Jia-Chen Gu Zi-Yi Dou Mohsen Fayyaz Pan Lu Kai-Wei Chang Nanyun Peng VLM 98 7 0 10 Oct 2024
Towards Synergistic, Generalized, and Efficient Dual-System for Robotic Manipulation Qingwen Bu Hongyang Li Li Chen Jisong Cai Jia Zeng Heming Cui Maoqing Yao Yu Qiao 87 7 0 10 Oct 2024
Towards Interpreting Visual Information Processing in Vision-Language Models Clement Neo Luke Ong Philip Torr Mor Geva David M. Krueger Fazl Barez 116 10 0 09 Oct 2024
MM-Ego: Towards Building Egocentric Multimodal LLMs for Video QA Hanrong Ye Haotian Zhang Erik Daxberger Lin Chen Zongyu Lin ... Haoxuan You Dan Xu Zhe Gan Jiasen Lu Yinfei Yang EgoV MLLM 115 14 0 09 Oct 2024
GLOV: Guided Large Language Models as Implicit Optimizers for Vision Language Models Muhammad Jehanzeb Mirza Mengjie Zhao Zhuoyuan Mao Sivan Doveh Wei Lin ... Yuki Mitsufuji Horst Possegger Rogerio Feris Leonid Karlinsky James Glass VLM 138 1 0 08 Oct 2024
Geometric Analysis of Reasoning Trajectories: A Phase Space Approach to Understanding Valid and Invalid Multi-Hop Reasoning in LLMs Javier Marin LRM 105 0 0 06 Oct 2024
Structure-Enhanced Protein Instruction Tuning: Towards General-Purpose Protein Understanding with LLMs Wei Wu Chao Wang L. Chen Mingze Yin Yiheng Zhu Kun Fu Jieping Ye Hui Xiong Zheng Wang 69 1 0 04 Oct 2024
SPARTUN3D: Situated Spatial Understanding of 3D World in Large Language Models Yue Zhang Zhiyang Xu Ying Shen Parisa Kordjamshidi Lifu Huang 89 6 0 04 Oct 2024
AuroraCap: Efficient, Performant Video Detailed Captioning and a New Benchmark Wenhao Chai Enxin Song Y. Du Chenlin Meng Vashisht Madhavan Omer Bar-Tal Jeng-Neng Hwang Saining Xie Christopher D. Manning 3DV 122 32 0 04 Oct 2024
Look Twice Before You Answer: Memory-Space Visual Retracing for Hallucination Mitigation in Multimodal Large Language Models Xin Zou Yizhou Wang Yibo Yan Yuanhuiyi Lyu Kening Zheng ... Junkai Chen Peijie Jiang Qingbin Liu Chang Tang Xuming Hu 110 8 0 04 Oct 2024
From Pixels to Tokens: Byte-Pair Encoding on Quantized Visual Modalities Wanpeng Zhang Zilong Xie Yicheng Feng Yijiang Li Xingrun Xing Sipeng Zheng Zongqing Lu MLLM 54 1 0 03 Oct 2024
Backdooring Vision-Language Models with Out-Of-Distribution Data Weimin Lyu Jiachen Yao Saumya Gupta Lu Pang Tao Sun Lingjie Yi Lijie Hu Haibin Ling Chao Chen VLM AAML 80 4 0 02 Oct 2024
Advancing Medical Radiograph Representation Learning: A Hybrid Pre-training Paradigm with Multilevel Semantic Granularity Hanqi Jiang Xixuan Hao Yuzhou Huang Chong Ma Jiaxun Zhang Yi Pan Ruimao Zhang MedIm 130 0 0 01 Oct 2024
FoAM: Foresight-Augmented Multi-Task Imitation Policy for Robotic Manipulation Litao Liu Wentao Wang Yifan Han Zhuoli Xie Pengfei Yi Junyan Li Yi Qin Wenzhao Lian 66 2 0 29 Sep 2024
CLIP-MoE: Towards Building Mixture of Experts for CLIP with Diversified Multiplet Upcycling Jihai Zhang Xiaoye Qu Tong Zhu Yu Cheng 72 8 0 28 Sep 2024