Kosmos-2: Grounding Multimodal Large Language Models to the World

26 June 2023

Papers citing "Kosmos-2: Grounding Multimodal Large Language Models to the World"

50 / 134 papers shown

Title
Understanding and Mitigating Toxicity in Image-Text Pretraining Datasets: A Case Study on LLaVA Karthik Reddy Kanjula Surya Guthikonda Nahid Alam Shayekh Bin Islam 26 0 0 09 May 2025
FG-CLIP: Fine-Grained Visual and Textual Alignment Chunyu Xie Bin Wang Fanjing Kong Jincheng Li Dawei Liang Gengshen Zhang Dawei Leng Yuhui Yin CLIP VLM 46 0 0 08 May 2025
Defining and Quantifying Creative Behavior in Popular Image Generators Aditi Ramaswamy Hana Chockler Melane Navaratnarajah 29 0 0 07 May 2025
LISAT: Language-Instructed Segmentation Assistant for Satellite Imagery Jerome Quenum Wen-Han Hsieh Tsung-Han Wu Ritwik Gupta Trevor Darrell David M. Chan MLLM VLM 54 0 0 05 May 2025
Unified Multimodal Understanding and Generation Models: Advances, Challenges, and Opportunities X. Zhang Jintao Guo Shanshan Zhao Minghao Fu Lunhao Duan Guo-Hua Wang Qing-Guo Chen Zhao Xu Weihua Luo Kaifu Zhang DiffM 74 0 0 05 May 2025
HapticVLM: VLM-Driven Texture Recognition Aimed at Intelligent Haptic Interaction Muhammad Haris Khan Miguel Altamirano Cabrera Dmitrii Iarchuk Yara Mahmoud Daria Trinitatova Issatay Tokmurziyev Dzmitry Tsetserukou VLM 48 0 0 05 May 2025
RESAnything: Attribute Prompting for Arbitrary Referring Segmentation Ruiqi Wang Hao Zhang VLM 56 0 0 03 May 2025
RoboGround: Robotic Manipulation with Grounded Vision-Language Priors Haifeng Huang Xinyi Chen Y. Chen H. Li Xiaoshen Han Z. Wang Tai Wang Jiangmiao Pang Zhou Zhao LM&Ro 80 0 0 30 Apr 2025
DREAM: Disentangling Risks to Enhance Safety Alignment in Multimodal Large Language Models J. Liu Hangyu Guo Ranjie Duan Xingyuan Bu Yancheng He ... Yingshui Tan Yanan Wu Jihao Gu Y. Li J. Zhu MLLM 142 0 0 25 Apr 2025
PRISM: A Unified Framework for Photorealistic Reconstruction and Intrinsic Scene Modeling Alara Dirik Tuanfeng Y. Wang Duygu Ceylan Stefanos Zafeiriou Anna Frühstück DiffM 47 0 0 19 Apr 2025
SmartFreeEdit: Mask-Free Spatial-Aware Image Editing with Complex Instruction Understanding Qianqian Sun Jixiang Luo Dell Zhang Xuelong Li DiffM 54 0 0 17 Apr 2025
Perception Encoder: The best visual embeddings are not at the output of the network Daniel Bolya Po-Yao (Bernie) Huang Peize Sun Jang Hyun Cho Andrea Madotto ... Shiyu Dong Nikhila Ravi Daniel Li Piotr Dollár Christoph Feichtenhofer ObjD VOS 103 0 0 17 Apr 2025
How Can Objects Help Video-Language Understanding? Zitian Tang Shijie Wang Junho Cho Jaewook Yoo Chen Sun 42 0 0 10 Apr 2025
On-device Sora: Enabling Training-Free Diffusion-based Text-to-Video Generation for Mobile Devices Bosung Kim Kyuhwan Lee Isu Jeong Jungmin Cheon Yeojin Lee Seulki Lee VGen 45 0 0 31 Mar 2025
Embedding Shift Dissection on CLIP: Effects of Augmentations on VLM's Representation Learning Ashim Dahal Saydul Akbar Murad Nick Rahimi VLM 45 0 0 30 Mar 2025
Aurelia: Test-time Reasoning Distillation in Audio-Visual LLMs Sanjoy Chowdhury Hanan Gani Nishit Anand Sayan Nag Ruohan Gao Mohamed Elhoseiny Salman Khan Dinesh Manocha LRM 49 0 0 29 Mar 2025
CTRL-O: Language-Controllable Object-Centric Visual Representation Learning Aniket Didolkar Andrii Zadaianchuk Rabiul Awal Maximilian Seitzer E. Gavves Aishwarya Agrawal OCL VLM 89 2 0 27 Mar 2025
Multimodal Chain-of-Thought Reasoning: A Comprehensive Survey Y. Wang Shengqiong Wu Y. Zhang William Yang Wang Ziwei Liu Jiebo Luo Hao Fei LRM 92 8 0 16 Mar 2025
Large-scale Pre-training for Grounded Video Caption Generation Evangelos Kazakos Cordelia Schmid Josef Sivic 59 0 0 13 Mar 2025
PlanGen: Towards Unified Layout Planning and Image Generation in Auto-Regressive Vision Language Models Runze He Bo Cheng Yuhang Ma Qingxiang Jia Shanyuan Liu Ao Ma Xiaoyu Wu Liebucha Wu Dawei Leng Yuhui Yin DiffM VLM 52 0 0 13 Mar 2025
Think Before You Segment: High-Quality Reasoning Segmentation with GPT Chain of Thoughts Shiu-hong Kao Yu-Wing Tai Chi-Keung Tang LRM MLLM 56 0 0 10 Mar 2025
WeGen: A Unified Model for Interactive Multimodal Generation as We Chat Zhipeng Huang Shaobin Zhuang Canmiao Fu Binxin Yang Ying Zhang Chong Sun Zhizheng Zhang Yali Wang Chen Li Zheng-Jun Zha DiffM 69 1 0 03 Mar 2025
R2-T2: Re-Routing in Test-Time for Multimodal Mixture-of-Experts Zhongyang Li Ziyue Li Tianyi Zhou MoE 51 0 0 27 Feb 2025
MQADet: A Plug-and-Play Paradigm for Enhancing Open-Vocabulary Object Detection via Multimodal Question Answering Caixiong Li Xiongwei Zhao Jinhang Zhang Xing Zhang Qihao Sun Zhou Wu ObjD MLLM VLM 51 0 0 23 Feb 2025
LOVA3: Learning to Visual Question Answering, Asking and Assessment Henry Hengyuan Zhao Pan Zhou Difei Gao Zechen Bai Mike Zheng Shou 77 8 0 21 Feb 2025
Contrastive Localized Language-Image Pre-Training Hong-You Chen Zhengfeng Lai H. Zhang X. Wang Marcin Eichner Keen You Meng Cao Bowen Zhang Y. Yang Zhe Gan CLIP VLM 68 7 0 20 Feb 2025
Magma: A Foundation Model for Multimodal AI Agents Jianwei Yang Reuben Tan Qianhui Wu Ruijie Zheng Baolin Peng ... Seonghyeon Ye Joel Jang Yuquan Deng Lars Liden Jianfeng Gao VLM AI4TS 122 9 0 18 Feb 2025
Decoder-Only LLMs are Better Controllers for Diffusion Models Ziyi Dong Yao Xiao Pengxu Wei Liang Lin DiffM 86 0 0 06 Feb 2025
Dynamic Scene Understanding from Vision-Language Representations Shahaf Pruss Morris Alper Hadar Averbuch-Elor OCL 164 0 0 20 Jan 2025
Myriad: Large Multimodal Model by Applying Vision Experts for Industrial Anomaly Detection Yuanze Li Haolin Wang Shihao Yuan Ming-Yu Liu Debin Zhao Yiwen Guo Chen Xu Guangming Shi Wangmeng Zuo 79 29 0 20 Jan 2025
Advancing General Multimodal Capability of Vision-language Models with Pyramid-descent Visual Position Encoding Z. Chen Mingxiao Li Z. Chen Nan Du Xiaolong Li Yuexian Zou 53 0 0 19 Jan 2025
DriveLM: Driving with Graph Visual Question Answering Chonghao Sima Katrin Renz Kashyap Chitta L. Chen Hanxue Zhang Chengen Xie Jens Beißwenger Ping Luo Andreas Geiger Hongyang Li 90 162 0 17 Jan 2025
Omni-RGPT: Unifying Image and Video Region-level Understanding via Token Marks Miran Heo Min-Hung Chen De-An Huang Sifei Liu Subhashree Radhakrishnan Seon Joo Kim Yu-Chun Wang Ryo Hachiuma ObjD VLM 159 2 0 14 Jan 2025
Sa2VA: Marrying SAM2 with LLaVA for Dense Grounded Understanding of Images and Videos Haobo Yuan X. Li Tao Zhang Zilong Huang Shilin Xu S. Ji Yunhai Tong Lu Qi Jiashi Feng Ming Yang VLM 94 11 0 07 Jan 2025
VisionLLM v2: An End-to-End Generalist Multimodal Large Language Model for Hundreds of Vision-Language Tasks Jiannan Wu Muyan Zhong Sen Xing Zeqiang Lai Zhaoyang Liu ... Lewei Lu Tong Lu Ping Luo Yu Qiao Jifeng Dai MLLM VLM LRM 99 48 0 03 Jan 2025
GPT4Scene: Understand 3D Scenes from Videos with Vision-Language Models Zhangyang Qi Zhixiong Zhang Ye Fang Jiaqi Wang Hengshuang Zhao 83 6 0 02 Jan 2025
HoVLE: Unleashing the Power of Monolithic Vision-Language Models with Holistic Vision-Language Embedding Chenxin Tao Shiqian Su X. Zhu Chenyu Zhang Zhe Chen ... Wenhai Wang Lewei Lu Gao Huang Yu Qiao Jifeng Dai MLLM VLM 104 2 0 20 Dec 2024
Next Patch Prediction for Autoregressive Visual Generation Yatian Pang Peng Jin Shuo Yang Bin Lin Bin Zhu ... Liuhan Chen Francis E. H. Tay Ser-Nam Lim Harry Yang Li Yuan 120 8 0 19 Dec 2024
Olympus: A Universal Task Router for Computer Vision Tasks Yuanze Lin Yunsheng Li Dongdong Chen Weijian Xu Ronald Clark Philip H. S. Torr VLM ObjD 185 0 0 12 Dec 2024
EgoPlan-Bench2: A Benchmark for Multimodal Large Language Model Planning in Real-World Scenarios Lu Qiu Yuying Ge Yi Chen Yixiao Ge Ying Shan Xihui Liu LLMAG LRM 96 5 0 05 Dec 2024
ROSE: Revolutionizing Open-Set Dense Segmentation with Patch-Wise Perceptual Large Multimodal Model Kunyang Han Yibo Hu Mengxue Qu Hailin Shi Yao Zhao Y. X. Wei MLLM VLM 3DV 85 1 0 29 Nov 2024
ChatRex: Taming Multimodal LLM for Joint Perception and Understanding Qing Jiang Gen Luo Yuqin Yang Yuda Xiong Yihao Chen Zhaoyang Zeng Tianhe Ren Lei Zhang VLM LRM 109 6 0 27 Nov 2024
Is 'Right' Right? Enhancing Object Orientation Understanding in Multimodal Large Language Models through Egocentric Instruction Tuning Ji Hyeok Jung Eun Tae Kim S. Kim Joo Ho Lee Bumsoo Kim Buru Chang VLM 178 0 0 24 Nov 2024
Efficient Transfer Learning for Video-language Foundation Models Haoxing Chen Zizheng Huang Y. Hong Yanshuo Wang Zhongcai Lyu Zhuoer Xu Jun Lan Zhangxuan Gu VLM 51 0 0 18 Nov 2024
GeoGround: A Unified Large Vision-Language Model for Remote Sensing Visual Grounding Y. Zhou Mengcheng Lan Xiang Li Yiping Ke Xue Jiang Litong Feng Qingyun Li Xue Yang Wayne Zhang ObjD VLM 116 4 0 16 Nov 2024
Spider: Any-to-Many Multimodal LLM Jinxiang Lai Jie Zhang Jun Liu Jian Li Xiaocheng Lu Song Guo MLLM 66 2 0 14 Nov 2024
VideoGLaMM: A Large Multimodal Model for Pixel-Level Visual Grounding in Videos Shehan Munasinghe Hanan Gani Wenqi Zhu Jiale Cao Eric P. Xing F. Khan Salman Khan MLLM VGen VLM 44 6 0 07 Nov 2024
AVHBench: A Cross-Modal Hallucination Benchmark for Audio-Visual Large Language Models Kim Sung-Bin Oh Hyun-Bin JungMok Lee Arda Senocak Joon Son Chung Tae-Hyun Oh MLLM VLM 40 3 0 23 Oct 2024
Do Vision-Language Models Represent Space and How? Evaluating Spatial Frame of Reference Under Ambiguities Zheyuan Zhang Fengyuan Hu Jayjun Lee Freda Shi Parisa Kordjamshidi Joyce Chai Ziqiao Ma 53 11 0 22 Oct 2024
Fine-Grained Verifiers: Preference Modeling as Next-token Prediction in Vision-Language Alignment Chenhang Cui An Zhang Yiyang Zhou Zhaorun Chen Gelei Deng Huaxiu Yao Tat-Seng Chua 68 4 0 18 Oct 2024