Osprey: Pixel Understanding with Visual Instruction Tuning

15 December 2023

Yuqian Yuan

Lei Zhang

Papers citing "Osprey: Pixel Understanding with Visual Instruction Tuning"

50 / 67 papers shown

Title
RESAnything: Attribute Prompting for Arbitrary Referring Segmentation Ruiqi Wang Hao Zhang VLM 56 0 0 03 May 2025
UniBiomed: A Universal Foundation Model for Grounded Biomedical Image Interpretation Linshan Wu Yuxiang Nie Sunan He Jiaxin Zhuang Hao Chen LM&MA MedIm 75 0 0 30 Apr 2025
DyFo: A Training-Free Dynamic Focus Visual Search for Enhancing LMMs in Fine-Grained Visual Understanding Geng Li Jinglin Xu Yunzhen Zhao Yuxin Peng ObjD 32 0 0 21 Apr 2025
EarthGPT-X: Enabling MLLMs to Flexibly and Comprehensively Understand Multi-Source Remote Sensing Imagery Wei Zhang Miaoxin Cai Yaqian Ning T. Zhang Yin Zhuang He Chen Jun Li Xuerui Mao 36 0 0 17 Apr 2025
Pixel-SAIL: Single Transformer For Pixel-Grounded Understanding Tao Zhang X. Li Zilong Huang Y. Li Weixian Lei XueQing Deng Shihao Chen S. Ji Jiashi Feng MLLM LRM 60 2 0 14 Apr 2025
OrderChain: A General Prompting Paradigm to Improve Ordinal Understanding Ability of MLLM Jinhong Wang Shuo Tong Jian Liu Dongqi Tang Weiqiang Wang Wentong Li Hongxia Xu Danny Chen J. Chen Jian Wu LRM 23 0 0 07 Apr 2025
URECA: Unique Region Caption Anything Sangbeom Lim J. Kim Heeji Yoon Jaewoo Jung Seungryong Kim 31 0 0 07 Apr 2025
InstructRestore: Region-Customized Image Restoration with Human Instructions S. Liu Jianqi Ma Lingchen Sun Xiangtao Kong Lei Zhang DiffM 44 0 0 31 Mar 2025
Towards Understanding How Knowledge Evolves in Large Vision-Language Models Sudong Wang Yuyao Zhang Yao Zhu Jianing Li Zizhe Wang Yi Liu Xiangyang Ji 137 0 0 31 Mar 2025
A Survey on Remote Sensing Foundation Models: From Vision to Multimodality Ziyue Huang Hongxi Yan Qiqi Zhan Shuai Yang Mingming Zhang Chenkai Zhang Yiming Lei Zeming Liu Qingjie Liu Yixuan Wang 46 0 0 28 Mar 2025
SceneSplat: Gaussian Splatting-based Scene Understanding with Vision-Language Pretraining Yue Li Qi Ma Runyi Yang Huapeng Li Mengjiao Ma ... E. Konukoglu Theo Gevers Luc Van Gool Martin R. Oswald Danda Pani Paudel 3DGS VLM 76 0 0 23 Mar 2025
GraspCorrect: Robotic Grasp Correction via Vision-Language Model-Guided Feedback Sungjae Lee Yeonjoo Hong Kwang In KIm 48 0 0 19 Mar 2025
MM-Spatial: Exploring 3D Spatial Understanding in Multimodal LLMs Erik Daxberger Nina Wenzel David Griffiths Haiming Gang Justin Lazarow ... Kai Kang Marcin Eichner Yuqing Yang Afshin Dehghan Peter Grasch 74 3 0 17 Mar 2025
SegAgent: Exploring Pixel Understanding Capabilities in MLLMs by Imitating Human Annotator Trajectories Muzhi Zhu Yuzhuo Tian Hao Chen Chunluan Zhou Qingpei Guo Y. Liu M. Yang Chunhua Shen MLLM VLM 72 0 0 11 Mar 2025
REF-VLM: Triplet-Based Referring Paradigm for Unified Visual Decoding Yan Tai Luhao Zhu Zhiqiang Chen Ynan Ding Yiying Dong Xiaohong Liu Guodong Guo MLLM ObjD 54 0 0 10 Mar 2025
Multi-Layer Visual Feature Fusion in Multimodal LLMs: Methods, Analysis, and Best Practices Junyan Lin Haoran Chen Yue Fan Yingqi Fan Xin Jin Hui Su Jinlan Fu Xiaoyu Shen 63 0 0 08 Mar 2025
Pixel-Level Reasoning Segmentation via Multi-turn Conversations Dexian Cai Xiaocui Yang Yongkang Liu Daling Wang Shi Feng Yifei Zhang Soujanya Poria LRM 82 0 0 13 Feb 2025
Omni-RGPT: Unifying Image and Video Region-level Understanding via Token Marks Miran Heo Min-Hung Chen De-An Huang Sifei Liu Subhashree Radhakrishnan Seon Joo Kim Yu-Chun Wang Ryo Hachiuma ObjD VLM 159 2 0 14 Jan 2025
Sa2VA: Marrying SAM2 with LLaVA for Dense Grounded Understanding of Images and Videos Haobo Yuan Xianrui Li Tao Zhang Zilong Huang Shilin Xu S. Ji Yunhai Tong Lu Qi Jiashi Feng Ming Yang VLM 96 11 0 07 Jan 2025
Visual Large Language Models for Generalized and Specialized Applications Yifan Li Zhixin Lai Wentao Bao Zhen Tan Anh Dao Kewei Sui Jiayi Shen Dong Liu Huan Liu Yu Kong VLM 88 11 0 06 Jan 2025
VisionLLM v2: An End-to-End Generalist Multimodal Large Language Model for Hundreds of Vision-Language Tasks Jiannan Wu Muyan Zhong Sen Xing Zeqiang Lai Zhaoyang Liu ... Lewei Lu Tong Lu Ping Luo Yu Qiao Jifeng Dai MLLM VLM LRM 99 48 0 03 Jan 2025
Vitron: A Unified Pixel-level Vision LLM for Understanding, Generating, Segmenting, Editing Hao Fei Shengqiong Wu H. Zhang Tat-Seng Chua Shuicheng Yan 64 38 0 31 Dec 2024
VideoRefer Suite: Advancing Spatial-Temporal Object Understanding with Video LLM Yuqian Yuan Hang Zhang Wentong Li Zesen Cheng Boqiang Zhang ... Deli Zhao Wenqiao Zhang Yueting Zhuang Jianke Zhu Lidong Bing 74 5 0 31 Dec 2024
CoF: Coarse to Fine-Grained Image Understanding for Multi-modal Large Language Models Yeyuan Wang D. Gao Bin Li Rujiao Long Lei Yi Xiaoyan Cai Libin Yang Jinxia Zhang Shanqing Yu Qi Xuan 78 1 0 22 Dec 2024
ROSE: Revolutionizing Open-Set Dense Segmentation with Patch-Wise Perceptual Large Multimodal Model Kunyang Han Yibo Hu Mengxue Qu Hailin Shi Yao Zhao Y. X. Wei MLLM VLM 3DV 88 1 0 29 Nov 2024
Detailed Object Description with Controllable Dimensions Xinran Wang H. Zhang Baoteng Li Kongming Liang Hao Sun Zhongjiang He Z. Ma Jun Guo 81 0 0 28 Nov 2024
ChatRex: Taming Multimodal LLM for Joint Perception and Understanding Qing Jiang Gen Luo Yuqin Yang Yuda Xiong Yihao Chen Zhaoyang Zeng Tianhe Ren Lei Zhang VLM LRM 109 6 0 27 Nov 2024
FINECAPTION: Compositional Image Captioning Focusing on Wherever You Want at Any Granularity Hang Hua Qing Liu Lingzhi Zhang Jing Shi Zhifei Zhang Yilin Wang Jianming Zhang Jiebo Luo CoGe VLM 92 6 0 23 Nov 2024
SEAGULL: No-reference Image Quality Assessment for Regions of Interest via Vision-Language Instruction Tuning Zhe Chen Juan Wang Wen Wang Sunhan Xu Hang Xiong ... Jian Guo Shuxun Wang C. Yuan Bing Li Weiming Hu VLM 43 1 0 15 Nov 2024
Griffon-G: Bridging Vision-Language and Vision-Centric Tasks via Large Multimodal Models Yufei Zhan Hongyin Zhao Yousong Zhu Fan Yang Ming Tang Jinqiao Wang MLLM 43 1 0 21 Oct 2024
Mitigating Object Hallucination via Concentric Causal Attention Yun Xing Yiheng Li Ivan Laptev Shijian Lu 45 18 0 21 Oct 2024
ForgeryGPT: Multimodal Large Language Model For Explainable Image Forgery Detection and Localization Jiawei Li Fanrui Zhang Jiaying Zhu Esther Sun Qiang Zhang Zheng-jun Zha MLLM 55 8 0 14 Oct 2024
DTVLT: A Multi-modal Diverse Text Benchmark for Visual Language Tracking Based on LLM Xuchen Li Shiyu Hu Xiaokun Feng Dailing Zhang Meiqi Wu Jing Zhang Kaiqi Huang 39 6 0 03 Oct 2024
MM1.5: Methods, Analysis & Insights from Multimodal LLM Fine-tuning Haotian Zhang Mingfei Gao Zhe Gan Philipp Dufter Nina Wenzel ... Haoxuan You Zirui Wang Afshin Dehghan Peter Grasch Yinfei Yang VLM MLLM 40 32 1 30 Sep 2024
EAGLE: Towards Efficient Arbitrary Referring Visual Prompts Comprehension for Multimodal Large Language Models Jiacheng Zhang Yang Jiao Shaoxiang Chen Jingjing Chen Yu-Gang Jiang 28 1 0 25 Sep 2024
Instruction-guided Multi-Granularity Segmentation and Captioning with Large Multimodal Model Li Zhou Xu Yuan Zenghui Sun Zikun Zhou Jingsong Lan VLM MLLM 127 3 0 20 Sep 2024
Image Segmentation in Foundation Model Era: A Survey Tianfei Zhou Fei Zhang Boyu Chang Wenguan Wang Ye Yuan E. Konukoglu Daniel Cremers VLM 42 4 0 23 Aug 2024
Sketch2Scene: Automatic Generation of Interactive 3D Game Scenes from User's Casual Sketches Yongzhi Xu Yonhon Ng Yifu Wang Inkyu Sa Yunfei Duan Yang Li Pan Ji Hongdong Li VGen 3DV 38 6 0 08 Aug 2024
ControlMLLM: Training-Free Visual Prompt Learning for Multimodal Large Language Models Ming-Kuan Wu Xinyue Cai Jiayi Ji Jiale Li Oucheng Huang Gen Luo Hao Fei Xiaoshuai Sun Rongrong Ji MLLM 49 7 0 31 Jul 2024
WTS: A Pedestrian-Centric Traffic Video Dataset for Fine-grained Spatial-Temporal Understanding Quan Kong Yuki Kawana Rajat Saini Ashutosh Kumar Jingjing Pan ... Yohei Ozao Balázs Opra D. Anastasiu Yoichi Sato Norimasa Kobori VGen 38 8 0 22 Jul 2024
EarthMarker: Visual Prompt Learning for Region-level and Point-level Remote Sensing Imagery Comprehension Wei Zhang Miaoxin Cai Tong Zhang Jun Li Zhuang Yin Xuerui Mao 63 5 0 18 Jul 2024
SPIN: Hierarchical Segmentation with Subpart Granularity in Natural Images Josh Myers-Dean Jarek Reynolds Brian Price Yifei Fan Danna Gurari 46 2 0 12 Jul 2024
Pseudo-RIS: Distinctive Pseudo-supervision Generation for Referring Image Segmentation Seonghoon Yu Paul Hongsuck Seo Jeany Son DiffM 57 4 0 10 Jul 2024
TokenPacker: Efficient Visual Projector for Multimodal LLM Wentong Li Yuqian Yuan Jian Liu Dongqi Tang Song Wang Jie Qin Jianke Zhu Lei Zhang MLLM 37 50 0 02 Jul 2024
OMG-LLaVA: Bridging Image-level, Object-level, Pixel-level Reasoning and Understanding Tao Zhang Xiangtai Li Hao Fei Haobo Yuan Shengqiong Wu Shunping Ji Chen Change Loy Shuicheng Yan LRM MLLM VLM 49 48 0 27 Jun 2024
SpatialBot: Precise Spatial Understanding with Vision Language Models Wenxiao Cai Yaroslav Ponomarenko Jianhao Yuan Xiaoqi Li Wankou Yang Hao Dong Bo-Lu Zhao VLM 53 28 0 19 Jun 2024
ClawMachine: Learning to Fetch Visual Tokens for Referential Comprehension Tianren Ma Lingxi Xie Yunjie Tian Boyu Yang Yuan Zhang 44 0 0 17 Jun 2024
Towards Semantic Equivalence of Tokenization in Multimodal LLM Shengqiong Wu Hao Fei Xiangtai Li Jiayi Ji Hanwang Zhang Tat-Seng Chua Shuicheng Yan MLLM 63 32 0 07 Jun 2024
SpatialRGPT: Grounded Spatial Reasoning in Vision Language Model An-Chieh Cheng Hongxu Yin Yang Fu Qiushan Guo Ruihan Yang Jan Kautz Xiaolong Wang Sifei Liu LRM 53 44 0 03 Jun 2024
The Evolution of Multimodal Model Architectures S. Wadekar Abhishek Chaurasia Aman Chadha Eugenio Culurciello 43 14 0 28 May 2024