v1v2 (latest)

Shikra: Unleashing Multimodal LLM's Referential Dialogue Magic

27 June 2023

ArXiv (abs)PDF HTML Github (778★)

Papers citing "Shikra: Unleashing Multimodal LLM's Referential Dialogue Magic"

50 / 163 papers shown

Title
Contrastive Localized Language-Image Pre-Training Hong-You Chen Zhengfeng Lai Hao Zhang Xiang Wang Marcin Eichner Keen You Meng Cao Bowen Zhang Yue Yang Zhe Gan CLIP VLM 124 10 0 20 Feb 2025
MindLLM: A Subject-Agnostic and Versatile Model for fMRI-to-Text Decoding Weikang Qiu Zheng Huang Haoyu Hu Aosong Feng Yujun Yan Rex Ying 99 0 0 18 Feb 2025
Vision-Language In-Context Learning Driven Few-Shot Visual Inspection Model Shiryu Ueno Yoshikazu Hayashi Shunsuke Nakatsuka Yusei Yamada Hiroaki Aizawa K. Kato MLLM VLM 201 0 0 13 Feb 2025
Mirage in the Eyes: Hallucination Attack on Multi-modal Large Language Models with Only Attention Sink Yining Wang Mi Zhang Junjie Sun Chenyue Wang Min Yang Hui Xue Jialing Tao Ranjie Duan Qingbin Liu 65 2 0 28 Jan 2025
InternVideo2.5: Empowering Video MLLMs with Long and Rich Context Modeling Yi Wang Xinhao Li Ziang Yan Yinan He Jiashuo Yu ... Kai Chen Wenhai Wang Yu Qiao Yali Wang Limin Wang 182 51 0 21 Jan 2025
Myriad: Large Multimodal Model by Applying Vision Experts for Industrial Anomaly Detection Yuanze Li Haolin Wang Shihao Yuan Ming-Yu Liu Debin Zhao Yiwen Guo Chen Xu Guangming Shi Wangmeng Zuo 162 33 0 20 Jan 2025
Omni-RGPT: Unifying Image and Video Region-level Understanding via Token Marks Miran Heo Min-Hung Chen De-An Huang Sifei Liu Subhashree Radhakrishnan Seon Joo Kim Yu-Chun Wang Ryo Hachiuma ObjD VLM 276 3 0 14 Jan 2025
Finer: Investigating and Enhancing Fine-Grained Visual Concept Recognition in Large Vision Language Models Jeonghwan Kim Heng Ji MLLM 106 2 0 08 Jan 2025
Visual Large Language Models for Generalized and Specialized Applications Yifan Li Zhixin Lai Wentao Bao Zhen Tan Anh Dao Kewei Sui Jiayi Shen Dong Liu Huan Liu Yu Kong VLM 174 15 0 06 Jan 2025
VisionLLM v2: An End-to-End Generalist Multimodal Large Language Model for Hundreds of Vision-Language Tasks Jiannan Wu Muyan Zhong Sen Xing Zeqiang Lai Zhaoyang Liu ... Lewei Lu Tong Lu Ping Luo Yu Qiao Jifeng Dai MLLM VLM LRM 363 59 0 03 Jan 2025
VideoRefer Suite: Advancing Spatial-Temporal Object Understanding with Video LLM Yuqian Yuan Hang Zhang Wentong Li Zesen Cheng Boqiang Zhang ... Deli Zhao Wenqiao Zhang Yueting Zhuang Jianke Zhu Lidong Bing 164 10 0 31 Dec 2024
Towards Visual Grounding: A Survey Linhui Xiao Xiaoshan Yang X. Lan Yaowei Wang Changsheng Xu ObjD 284 5 0 31 Dec 2024
Vitron: A Unified Pixel-level Vision LLM for Understanding, Generating, Segmenting, Editing Hao Fei Shengqiong Wu Hao Zhang Tat-Seng Chua Shuicheng Yan 190 42 0 31 Dec 2024
Aria-UI: Visual Grounding for GUI Instructions Yuhao Yang Yue Wang Dongxu Li Ziyang Luo Bei Chen Chenyu Huang Junnan Li LM&Ro LLMAG 176 33 0 20 Dec 2024
HoVLE: Unleashing the Power of Monolithic Vision-Language Models with Holistic Vision-Language Embedding Chenxin Tao Shiqian Su X. Zhu Chenyu Zhang Zhe Chen ... Wenhai Wang Lewei Lu Gao Huang Yu Qiao Jifeng Dai MLLM VLM 256 2 0 20 Dec 2024
Nullu: Mitigating Object Hallucinations in Large Vision-Language Models via HalluSpace Projection Le Yang Ziwei Zheng Boxu Chen Zhengyu Zhao Chenhao Lin Chao Shen VLM 317 7 0 18 Dec 2024
Olympus: A Universal Task Router for Computer Vision Tasks Yuanze Lin Yunsheng Li Dongdong Chen Weijian Xu Ronald Clark Philip Torr VLM ObjD 548 1 0 12 Dec 2024
Paint Outside the Box: Synthesizing and Selecting Training Data for Visual Grounding Zilin Du Haoxin Li Jianfei Yu Boyang Li 498 0 0 01 Dec 2024
ROSE: Revolutionizing Open-Set Dense Segmentation with Patch-Wise Perceptual Large Multimodal Model Kunyang Han Yibo Hu Mengxue Qu Hailin Shi Yao Zhao Y. X. Wei MLLM VLM 3DV 271 1 0 29 Nov 2024
ChatRex: Taming Multimodal LLM for Joint Perception and Understanding Qing Jiang Gen Luo Yuqin Yang Yuda Xiong Yihao Chen Zhaoyang Zeng Tianhe Ren Lei Zhang VLM LRM 216 10 0 27 Nov 2024
Autonomous Imagination: Closed-Loop Decomposition of Visual-to-Textual Conversion in Visual Reasoning for Multimodal Large Language Models Qingbin Liu Yumeng Li Boyuan Xiao Yichang Jian Ziang Qin Tianjia Shao Yao-Xiang Ding Kun Zhou LRM MLLM 211 3 0 27 Nov 2024
Beyond Walking: A Large-Scale Image-Text Benchmark for Text-based Person Anomaly Search Shuyu Yang Yaxiong Wang Li Zhu Zhedong Zheng 214 7 0 26 Nov 2024
Beyond Sight: Towards Cognitive Alignment in LVLM via Enriched Visual Knowledge Yaqi Zhao Yuanyang Yin Lin Li Mingan Lin Victor Shea-Jay Huang Siwei Chen Xin Wu Baoqun Yin Guosheng Dong Wentao Zhang 136 1 0 25 Nov 2024
Devils in Middle Layers of Large Vision-Language Models: Interpreting, Detecting and Mitigating Object Hallucinations via Attention Lens Zhangqi Jiang Junkai Chen Beier Zhu Tingjin Luo Yankun Shen Xu Yang 180 7 0 23 Nov 2024
From Holistic to Localized: Local Enhanced Adapters for Efficient Visual Instruction Fine-Tuning Pengkun Jiao Bin Zhu Jingjing Chen Chong-Wah Ngo Yu-Gang Jiang VLM OffRL 168 0 0 19 Nov 2024
GeoGround: A Unified Large Vision-Language Model for Remote Sensing Visual Grounding Yimiao Zhou Mengcheng Lan Xiang Li Yiping Ke Yiping Ke Xue Jiang Qingyun Li Xue Yang Wayne Zhang ObjD VLM 269 7 0 16 Nov 2024
VideoGLaMM: A Large Multimodal Model for Pixel-Level Visual Grounding in Videos Shehan Munasinghe Hanan Gani Wenqi Zhu Jiale Cao Eric P. Xing Fahad Shahbaz Khan Salman Khan MLLM VGen VLM 127 9 0 07 Nov 2024
Unified Triplet-Level Hallucination Evaluation for Large Vision-Language Models J. Wu Tsz Ting Chung Kai Chen Dit-Yan Yeung LRM VLM 134 4 0 30 Oct 2024
GiVE: Guiding Visual Encoder to Perceive Overlooked Information Junjie Li Jianghong Ma Xiaofeng Zhang Yuhang Li Jianyang Shi 128 1 0 26 Oct 2024
PyramidDrop: Accelerating Your Large Vision-Language Models via Pyramid Visual Redundancy Reduction Long Xing Qidong Huang Xiaoyi Dong Jiajie Lu Pan Zhang ... Yuhang Cao Zeang Sheng Jiaqi Wang Feng Wu Dahua Lin VLM 133 46 0 22 Oct 2024
Do Vision-Language Models Represent Space and How? Evaluating Spatial Frame of Reference Under Ambiguities Zheyuan Zhang Fengyuan Hu Jayjun Lee Freda Shi Parisa Kordjamshidi Joyce Chai Ziqiao Ma 157 15 0 22 Oct 2024
TransAgent: Transfer Vision-Language Foundation Models with Heterogeneous Agent Collaboration Yiwei Guo Shaobin Zhuang Kunchang Li Yu Qiao Yali Wang VLM CLIP 137 1 0 16 Oct 2024
OneRef: Unified One-tower Expression Grounding and Segmentation with Mask Referring Modeling Linhui Xiao Xiaoshan Yang Fang Peng Yaowei Wang Changsheng Xu ObjD 126 7 0 10 Oct 2024
MRAG-Bench: Vision-Centric Evaluation for Retrieval-Augmented Multimodal Models Wenbo Hu Jia-Chen Gu Zi-Yi Dou Mohsen Fayyaz Pan Lu Kai-Wei Chang Nanyun Peng VLM 148 8 0 10 Oct 2024
Navigating the Digital World as Humans Do: Universal Visual Grounding for GUI Agents Boyu Gou Ruohan Wang Boyuan Zheng Yanan Xie Cheng Chang Yiheng Shu Huan Sun Yu Su LM&Ro LLMAG 255 96 0 07 Oct 2024
EMMA: Efficient Visual Alignment in Multi-Modal LLMs Sara Ghazanfari Alexandre Araujo Prashanth Krishnamurthy Siddharth Garg Farshad Khorrami VLM 81 2 0 02 Oct 2024
When SAM2 Meets Video Camouflaged Object Segmentation: A Comprehensive Evaluation and Adaptation Yuli Zhou Guolei Sun Yawei Li Guo-Sen Xie Luca Benini Ender Konukoglu 160 5 0 27 Sep 2024
LLaVA-3D: A Simple yet Effective Pathway to Empowering LMMs with 3D-awareness Chenming Zhu Tai Wang Wenwei Zhang Jiangmiao Pang Xihui Liu 248 52 0 26 Sep 2024
FineCops-Ref: A new Dataset and Task for Fine-Grained Compositional Referring Expression Comprehension Junzhuo Liu Xiaohu Yang Weiwei Li Peng Wang ObjD 139 5 0 23 Sep 2024
MaPPER: Multimodal Prior-guided Parameter Efficient Tuning for Referring Expression Comprehension Ting Liu Zunnan Xu Yue Hu Liangtao Shi Zhiqiang Wang Quanjun Yin 155 3 0 20 Sep 2024
An overview of domain-specific foundation model: key technologies, applications and challenges Haolong Chen Hanzhi Chen Zijian Zhao Kaifeng Han Guangxu Zhu Yichen Zhao Ying Du Wei Xu Qingjiang Shi ALM VLM 129 5 0 06 Sep 2024
IAA: Inner-Adaptor Architecture Empowers Frozen Large Language Model with Multimodal Capabilities Bin Wang Chunyu Xie Dawei Leng Yuhui Yin MLLM 179 1 0 23 Aug 2024
ParGo: Bridging Vision-Language with Partial and Global Views An-Lan Wang Bin Shan Wei Shi Kun-Yu Lin Xiang Fei Guozhi Tang Lei Liao Jingqun Tang Can Huang Wei-Shi Zheng MLLM VLM 185 17 0 23 Aug 2024
CLIP-DPO: Vision-Language Models as a Source of Preference for Fixing Hallucinations in LVLMs Yassine Ouali Adrian Bulat Brais Martínez Georgios Tzimiropoulos VLM MLLM 107 25 0 19 Aug 2024
Visual Agents as Fast and Slow Thinkers Guangyan Sun Mingyu Jin Zhenting Wang Cheng-Long Wang Siqi Ma Qifan Wang Ying Nian Wu Ying Nian Wu Dongfang Liu Dongfang Liu LLMAG LRM 222 19 0 16 Aug 2024
Self-Introspective Decoding: Alleviating Hallucinations for Large Vision-Language Models Fushuo Huo Wenchao Xu Zhong Zhang Yining Qi Zhicheng Chen Peilin Zhao VLM MLLM 212 31 0 04 Aug 2024
Evaluating SAM2's Role in Camouflaged Object Detection: From SAM to SAM2 Lv Tang Bo Li VLM 74 7 0 31 Jul 2024
ControlMLLM: Training-Free Visual Prompt Learning for Multimodal Large Language Models Ming-Kuan Wu Xinyue Cai Jiayi Ji Jiale Li Oucheng Huang Gen Luo Hao Fei Xiaoshuai Sun Rongrong Ji MLLM 160 13 0 31 Jul 2024
XMeCap: Meme Caption Generation with Sub-Image Adaptability Yuyan Chen Songzhou Yan Zhihong Zhu Zhixu Li Yanghua Xiao VLM 125 12 0 24 Jul 2024
X-Former: Unifying Contrastive and Reconstruction Learning for MLLMs S. Swetha Jinyu Yang T. Neiman Mamshad Nayeem Rizve Son Tran Benjamin Z. Yao Trishul Chilimbi Mubarak Shah 112 2 0 18 Jul 2024