LLaVA-NeXT-Interleave: Tackling Multi-image, Video, and 3D in Large Multimodal Models

10 July 2024

Feng Li

Renrui Zhang

Hao Zhang

Yuanhan Zhang

Bo Li

Wei Li

Zejun Ma

Papers citing "LLaVA-NeXT-Interleave: Tackling Multi-image, Video, and 3D in Large Multimodal Models"

50 / 154 papers shown

Title
Unveiling the Potential of Vision-Language-Action Models with Open-Ended Multimodal Instructions Wei Zhao Gongsheng Li Zhefei Gong Pengxiang Ding H. Zhao Donglin Wang LM&Ro 19 0 0 16 May 2025
Exploring Implicit Visual Misunderstandings in Multimodal Large Language Models through Attention Analysis Pengfei Wang Guohai Xu Weinong Wang Junjie Yang Jie Lou Yunhua Xue 24 0 0 15 May 2025
Does Feasibility Matter? Understanding the Impact of Feasibility on Synthetic Training Data Yiwen Liu Jessica Bader Jae Myung Kim DiffM 16 0 0 15 May 2025
Flash-VL 2B: Optimizing Vision-Language Model Performance for Ultra-Low Latency and High Throughput Bo Zhang Shuo Li Runhe Tian Yang Yang Jixin Tang Jinhao Zhou Lin Ma VLM 27 0 0 14 May 2025
PlaceIt3D: Language-Guided Object Placement in Real 3D Scenes Ahmed Abdelreheem Filippo Aleotti Jamie Watson Z. Qureshi Abdelrahman Eldesokey Peter Wonka Gabriel J. Brostow Sara Vicente Guillermo Garcia-Hernando DiffM 59 0 0 08 May 2025
Transferable Adversarial Attacks on Black-Box Vision-Language Models Kai Hu Weichen Yu L. Zhang Alexander Robey Andy Zou Chengming Xu Haoqi Hu Matt Fredrikson AAML VLM 64 0 0 02 May 2025
TEMPURA: Temporal Event Masked Prediction and Understanding for Reasoning in Action Jen-Hao Cheng Vivian Wang Huayu Wang Huapeng Zhou Yi-Hao Peng ... Wenhao Chai Yi-Ling Chen Vibhav Vineet Qin Cai Jenq-Neng Hwang AI4TS 142 0 0 02 May 2025
Zoomer: Adaptive Image Focus Optimization for Black-box MLLM Jiaxu Qian Chendong Wang Y. Yang Chaoyun Zhang Huiqiang Jiang ... Saravan Rajmohan Dongmei Zhang Y. Yang Qi Zhang Lili Qiu VLM 81 0 0 30 Apr 2025
Benchmarking Multimodal Mathematical Reasoning with Explicit Visual Dependency Zhikai Wang Jiashuo Sun W. Zhang Zhiqiang Hu Xin Li F. Wang Deli Zhao VLM LRM 75 0 0 24 Apr 2025
FRAG: Frame Selection Augmented Generation for Long Video and Long Document Understanding De-An Huang Subhashree Radhakrishnan Zhiding Yu Jan Kautz VGen VLM 78 0 0 24 Apr 2025
Vidi: Large Multimodal Models for Video Understanding and Editing Vidi Team Celong Liu Chia-Wen Kuo Dawei Du Fan Chen ... Wen Zhong Xiaohui Shen Xin Gu Xing Mei Xueqiong Qu 67 0 0 22 Apr 2025
T2VShield: Model-Agnostic Jailbreak Defense for Text-to-Video Models Siyuan Liang Jiayang Liu Jiecheng Zhai Tianmeng Fang Rongcheng Tu A. Liu Xiaochun Cao Dacheng Tao VGen 61 0 0 22 Apr 2025
Ask2Loc: Learning to Locate Instructional Visual Answers by Asking Questions Chang Zong Bin Li Shoujun Zhou Jian Wan Lei Zhang 129 0 0 22 Apr 2025
MR. Video: "MapReduce" is the Principle for Long Video Understanding Ziqi Pang Yu-xiong Wang VLM 45 0 0 22 Apr 2025
DyFo: A Training-Free Dynamic Focus Visual Search for Enhancing LMMs in Fine-Grained Visual Understanding Geng Li Jinglin Xu Yunzhen Zhao Yuxin Peng ObjD 32 0 0 21 Apr 2025
Video-MMLU: A Massive Multi-Discipline Lecture Understanding Benchmark Enxin Song Wenhao Chai Weili Xu Jianwen Xie Yuxuan Liu Gaoang Wang 62 0 0 20 Apr 2025
OmniV-Med: Scaling Medical Vision-Language Model for Universal Visual Understanding Songtao Jiang Yuan Wang Sibo Song Y. Zhang Zijie Meng Bohan Lei Jian Wu Jimeng Sun Zuozhu Liu MedIm VLM 37 0 0 20 Apr 2025
VideoPASTA: 7K Preference Pairs That Matter for Video-LLM Alignment Yogesh Kulkarni Pooyan Fazli 38 0 0 18 Apr 2025
VistaDPO: Video Hierarchical Spatial-Temporal Direct Preference Optimization for Large Video Models Haojian Huang Haodong Chen Shengqiong Wu Meng Luo Jinlan Fu Xinya Du H. Zhang Hao Fei AI4TS 151 0 0 17 Apr 2025
AnomalyR1: A GRPO-based End-to-end MLLM for Industrial Anomaly Detection Yuhao Chao Jie Liu J. Tang Gangshan Wu 35 1 0 16 Apr 2025
Mavors: Multi-granularity Video Representation for Multimodal Large Language Model Yang Shi Jiaheng Liu Yushuo Guan Z. Wu Y. Zhang ... Bohan Zeng W. Zhang Fuzheng Zhang Wenjing Yang Di Zhang VGen VLM 71 0 0 14 Apr 2025
TAMP: Token-Adaptive Layerwise Pruning in Multimodal Large Language Models Jaewoo Lee Keyang Xuan Chanakya Ekbote Sandeep Polisetty Yi Ren Fung Paul Pu Liang VLM 37 0 0 14 Apr 2025
The Structural Safety Generalization Problem Julius Broomfield Tom Gibbs Ethan Kosak-Hine George Ingebretsen Tia Nasir Jason Zhang Reihaneh Iranmanesh Sara Pieri Reihaneh Rabbany Kellin Pelrine AAML 29 0 0 13 Apr 2025
Mimic In-Context Learning for Multimodal Tasks Yuchu Jiang Jiale Fu Chenduo Hao Xinting Hu Yingzhe Peng Xin Geng Xu Yang 27 0 0 11 Apr 2025
TokenFocus-VQA: Enhancing Text-to-Image Alignment with Position-Aware Focus and Multi-Perspective Aggregations on LVLMs Zijian Zhang Xuhui Zheng X. Wu Chong Peng Xuezhi Cao 32 0 0 10 Apr 2025
SF2T: Self-supervised Fragment Finetuning of Video-LLMs for Fine-Grained Understanding Yangliu Hu Zikai Song Na Feng Yawei Luo Junqing Yu Yi-Ping Phoebe Chen Wei Yang 33 0 0 10 Apr 2025
REVEAL: Relation-based Video Representation Learning for Video-Question-Answering Sofian Chaybouti Walid Bousselham Moritz Wolter Hilde Kuehne 110 0 0 07 Apr 2025
Aligned Better, Listen Better for Audio-Visual Large Language Models Yuxin Guo Shuailei Ma Shijie Ma Xiaoyi Bao Chen-Wei Xie Kecheng Zheng Tingyu Weng Siyang Sun Yun Zheng Wei Zou MLLM AuLLM 62 2 0 02 Apr 2025
Overcoming Vocabulary Constraints with Pixel-level Fallback Jonas F. Lotz Hendra Setiawan Stephan Peitz Yova Kementchedjhieva 43 0 0 02 Apr 2025
Q-Adapt: Adapting LMM for Visual Quality Assessment with Progressive Instruction Tuning Yiting Lu X. Li H. Wu Bingchen Li Weisi Lin Zhibo Chen 37 1 0 02 Apr 2025
Improved Visual-Spatial Reasoning via R1-Zero-Like Training Zhenyi Liao Qingsong Xie Yanhao Zhang Zijian Kong Haonan Lu Zhenyu Yang Zhijie Deng ReLM VLM LRM 101 0 1 01 Apr 2025
Short-video Propagation Influence Rating: A New Real-world Dataset and A New Large Graph Model Dizhan Xue Jing Cui Shengsheng Qian Chuanrui Hu Changsheng Xu 39 0 0 31 Mar 2025
OmniMMI: A Comprehensive Multi-modal Interaction Benchmark in Streaming Video Contexts Y. Wang Y. Wang Bo Chen Tong Wu Dongyan Zhao Zilong Zheng VLM MLLM 63 1 0 29 Mar 2025
Unicorn: Text-Only Data Synthesis for Vision Language Model Training Xiaomin Yu Pengxiang Ding Wenjie Zhang Siteng Huang Songyang Gao Chengwei Qin Kejian Wu Zhaoxin Fan Ziyue Qiao Donglin Wang MLLM SyDa 67 0 0 28 Mar 2025
VBench-2.0: Advancing Video Generation Benchmark Suite for Intrinsic Faithfulness Dian Zheng Ziqi Huang Hongbo Liu Kai Zou Yinan He ... Y. Zhang Jingwen He Wei-Shi Zheng Yu Qiao Ziwei Liu EGVM VGen 48 5 0 27 Mar 2025
StyleMotif: Multi-Modal Motion Stylization using Style-Content Cross Fusion Ziyu Guo Young Yoon Lee Joseph Liu Yizhak Ben-Shabat Victor Zordan Mubbasir Kapadia DiffM VGen 73 0 0 27 Mar 2025
Fwd2Bot: LVLM Visual Token Compression with Double Forward Bottleneck Adrian Bulat Yassine Ouali Georgios Tzimiropoulos 143 0 0 27 Mar 2025
Instruction-Oriented Preference Alignment for Enhancing Multi-Modal Comprehension Capability of MLLMs Zitian Wang Yue Liao Kang Rong Fengyun Rao Yibo Yang Si Liu 70 0 0 26 Mar 2025
FALCONEye: Finding Answers and Localizing Content in ONE-hour-long videos with multi-modal LLMs Carlos Plou Cesar Borja Ruben Martinez-Cantin Ana C. Murillo 61 0 0 25 Mar 2025
PAVE: Patching and Adapting Video Large Language Models Zhuoming Liu Yiquan Li Khoi Duc Nguyen Yiwu Zhong Yin Li KELM LRM 83 0 0 25 Mar 2025
Improved Alignment of Modalities in Large Vision Language Models Kartik Jangra Aman Kumar Singh Yashwani Mann Geetanjali Rathee VLM 50 0 0 25 Mar 2025
RGB-Th-Bench: A Dense benchmark for Visual-Thermal Understanding of Vision Language Models Mehdi Moshtaghi Siavash H. Khajavi Joni Pajarinen VLM 49 0 0 25 Mar 2025
A Simple yet Effective Layout Token in Large Language Models for Document Understanding Zhaoqing Zhu Chuwei Luo Zirui Shao Feiyu Gao Hangdi Xing Qi Zheng Ji Zhang 50 0 0 24 Mar 2025
What can Off-the-Shelves Large Multi-Modal Models do for Dynamic Scene Graph Generation? Xuanming Cui Jaiminkumar Ashokbhai Bhoi Chionh Wei Peng Adriel Kuek Ser-Nam Lim 48 0 0 20 Mar 2025
When Less is Enough: Adaptive Token Reduction for Efficient Image Representation Eduard Allakhverdov Elizaveta Goncharova Andrey Kuznetsov 42 0 0 20 Mar 2025
MathFlow: Enhancing the Perceptual Flow of MLLMs for Visual Mathematical Problems Felix Chen Hangjie Yuan Yunqiu Xu Tao Feng Jun Cen Pengwei Liu Zeying Huang Yi Yang LRM 44 1 0 19 Mar 2025
CoSpace: Benchmarking Continuous Space Perception Ability for Vision-Language Models Yiqi Zhu Z. Wang C. Zhang Peng Li Yang Liu CoGe VLM 68 0 0 18 Mar 2025
Can Large Vision Language Models Read Maps Like a Human? Shuo Xing Zezhou Sun Shuangyu Xie Kaiyuan Chen Yanjia Huang Yuping Wang Jiachen Li Dezhen Song Zhengzhong Tu 68 2 0 18 Mar 2025
Aligning Multimodal LLM with Human Preference: A Survey Tao Yu Y. Zhang Chaoyou Fu Junkang Wu Jinda Lu ... Qingsong Wen Z. Zhang Yan Huang Liang Wang T. Tan 161 2 0 18 Mar 2025
Identifying and Mitigating Position Bias of Multi-image Vision-Language Models Xinyu Tian Shu Zou Zhaoyuan Yang Jing Zhang 63 0 0 18 Mar 2025