Visual Instruction Tuning

17 April 2023

Papers citing "Visual Instruction Tuning"

50 / 3,278 papers shown

Title
Social-LLaVA: Enhancing Robot Navigation through Human-Language Reasoning in Social Spaces Amirreza Payandeh Daeun Song Mohammad Nazeri Jing Liang Praneel Mukherjee Amir Hossain Raj Yangzhe Kong Dinesh Manocha Xuesu Xiao LM&Ro LRM 79 5 0 17 Jan 2025
AudioBERT: Audio Knowledge Augmented Language Model Hyunjong Ok Suho Yoo Jaeho Lee AuLLM RALM VLM 53 0 0 17 Jan 2025
Playing Devil's Advocate: Unmasking Toxicity and Vulnerabilities in Large Vision-Language Models Abdulkadir Erol Trilok Padhi Agnik Saha Ugur Kursuncu Mehmet Emin Aktas 58 1 0 17 Jan 2025
SD-Eval: A Benchmark Dataset for Spoken Dialogue Understanding Beyond Words Junyi Ao Yuancheng Wang Xiaohai Tian Dekun Chen Jingyang Zhang Lu Lu Yansen Wang Haizhou Li Zhikai Wu AuLLM 90 19 0 17 Jan 2025
DriveLM: Driving with Graph Visual Question Answering Chonghao Sima Katrin Renz Kashyap Chitta Lawrence Yunliang Chen Hanxue Zhang Chengen Xie Jens Beißwenger Ping Luo Andreas Geiger Hongyang Li 108 170 0 17 Jan 2025
The Devil is in Temporal Token: High Quality Video Reasoning Segmentation Sitong Gong Yunzhi Zhuge Lu Zhang Zheng Yang Pingping Zhang Huchuan Lu 49 0 0 15 Jan 2025
Omni-RGPT: Unifying Image and Video Region-level Understanding via Token Marks Miran Heo Min-Hung Chen De-An Huang Sifei Liu Subhashree Radhakrishnan Seon Joo Kim Yu-Chun Wang Ryo Hachiuma ObjD VLM 170 2 0 14 Jan 2025
A Heterogeneous Multimodal Graph Learning Framework for Recognizing User Emotions in Social Networks Sree Bhattacharyya Shuhua Yang James Z. Wang 43 0 0 13 Jan 2025
Zero-Shot Scene Understanding for Automatic Target Recognition Using Large Vision-Language Models Y. Ranasinghe Vibashan Vs James Uplinger C. D. Melo Vishal M. Patel 47 0 0 13 Jan 2025
GestLLM: Advanced Hand Gesture Interpretation via Large Language Models for Human-Robot Interaction Oleg Kobzarev Artem Lykov Dzmitry Tsetserukou VLM 45 1 0 13 Jan 2025
LEO: Boosting Mixture of Vision Encoders for Multimodal Large Language Models Mozhgan Nasr Azadani James Riddell Sean Sedwards Krzysztof Czarnecki MLLM VLM 62 3 0 13 Jan 2025
Using Pre-trained LLMs for Multivariate Time Series Forecasting Malcolm Wolff Shenghao Yang Kari Torkkola Michael W. Mahoney AI4TS AIFin 59 1 0 10 Jan 2025
VideoAuteur: Towards Long Narrative Video Generation Junfei Xiao Feng Cheng Lu Qi Liangke Gui Jiepeng Cen Zhibei Ma Alan Yuille Lu Jiang VGen 72 2 0 10 Jan 2025
AgroGPT: Efficient Agricultural Vision-Language Model with Expert Tuning Muhammad Awais Ali Husain Salem Abdulla Alharthi Amandeep Kumar Hisham Cholakkal Rao Muhammad Anwer VLM 65 3 0 10 Jan 2025
OneLLM: One Framework to Align All Modalities with Language Jiaming Han Kaixiong Gong Yiyuan Zhang Jiaqi Wang Kaipeng Zhang Dahua Lin Yu Qiao Peng Gao Xiangyu Yue MLLM 108 111 0 10 Jan 2025
MedCoDi-M: A Multi-Prompt Foundation Model for Multimodal Medical Data Generation Daniele Molino Francesco Di Feola E. Faiella Deborah Fazzini D. Santucci Linlin Shen V. Guarrasi Paolo Soda SyDa MedIm 49 0 0 10 Jan 2025
Generative AI for Cel-Animation: A Survey Yunlong Tang Junjia Guo Pinxin Liu Zhiyuan Wang Hang Hua ... Jing Bi Mingqian Feng Xuzhao Li Zeliang Zhang Chenliang Xu VGen 96 8 0 08 Jan 2025
iServe: An Intent-based Serving System for LLMs Dimitrios Liakopoulos Tianrui Hu Prasoon Sinha N. Yadwadkar VLM 280 0 0 08 Jan 2025
Video-of-Thought: Step-by-Step Video Reasoning from Perception to Cognition Hao Fei Shengqiong Wu Wei Ji Hao Zhang Hao Fei Mong Li Lee Wynne Hsu LRM VGen 63 68 0 08 Jan 2025
Learning the Language of Protein Structure Benoit Gaujac Jérémie Donà Liviu Copoiu Timothy Atkinson Thomas Pierrot Thomas D. Barrett 78 11 0 08 Jan 2025
ConceptMaster: Multi-Concept Video Customization on Diffusion Transformer Models Without Test-Time Tuning Yuzhou Huang Ziyang Yuan Quande Liu Qiulin Wang Xintao Wang Ruimao Zhang Pengfei Wan Di Zhang Kun Gai VGen DiffM 61 10 0 08 Jan 2025
OmniManip: Towards General Robotic Manipulation via Object-Centric Interaction Primitives as Spatial Constraints Mingjie Pan Jiyao Zhang Tianshu Wu Yinghao Zhao Wenlong Gao Hao Dong LM&Ro 66 8 0 08 Jan 2025
Rare-to-Frequent: Unlocking Compositional Generation Power of Diffusion Models on Rare Concepts with LLM Guidance Dongmin Park Sebin Kim Taehong Moon Minkyu Kim Kangwook Lee Jaewoong Cho DiffM CoGe 75 2 0 08 Jan 2025
Clinical Insights: A Comprehensive Review of Language Models in Medicine Nikita Neveditsin Pawan Lingras V. Mago LM&MA 63 4 0 08 Jan 2025
Feedback-Driven Vision-Language Alignment with Minimal Human Supervision Giorgio Giannone Ruoteng Li Qianli Feng Evgeny Perevodchikov Rui Chen Aleix M. Martinez VLM 71 0 0 08 Jan 2025
URSA: Understanding and Verifying Chain-of-thought Reasoning in Multimodal Mathematics Ruilin Luo Zhuofan Zheng Yifan Wang Yiyao Yu Xinzhe Ni Zicheng Lin Jin Zeng Yujiu Yang LRM 83 14 0 08 Jan 2025
Sa2VA: Marrying SAM2 with LLaVA for Dense Grounded Understanding of Images and Videos Haobo Yuan Xianrui Li Tao Zhang Zilong Huang Shilin Xu S. Ji Yunhai Tong Lu Qi Jiashi Feng Ming-Hsuan Yang VLM 96 12 0 07 Jan 2025
LLaVA-Mini: Efficient Image and Video Large Multimodal Models with One Vision Token Shaolei Zhang Qingkai Fang Zhe Yang Yang Feng MLLM VLM 84 28 0 07 Jan 2025
MotionBench: Benchmarking and Improving Fine-grained Video Motion Understanding for Vision Language Models Wenyi Hong Yean Cheng Zheng Yang Weihan Wang Lefan Wang Xiaotao Gu Shiyu Huang Yuxiao Dong J. Tang CoGe VLM 75 4 0 06 Jan 2025
Visual Large Language Models for Generalized and Specialized Applications Yifan Li Zhixin Lai Wentao Bao Zhen Tan Anh Dao Kewei Sui Jiayi Shen Dong Liu Huan Liu Yu Kong VLM 91 12 0 06 Jan 2025
Automated Generation of Challenging Multiple-Choice Questions for Vision Language Model Evaluation Yuhui Zhang Yuchang Su Yiming Liu Xiaohan Wang James Burgess ... Josiah Aklilu Alejandro Lozano Anjiang Wei Ludwig Schmidt Serena Yeung-Levy 66 3 0 06 Jan 2025
Multi-LLM Collaborative Caption Generation in Scientific Documents Jaeyoung Kim J. B. Lee Hong-Jun Choi Ting-Yao Hsu Chieh-Yang Huang ... Ryan Rossi Tong Yu C. Lee Giles Ting-Hao 'Kenneth' Huang S. Choi 37 2 0 05 Jan 2025
FedRSClip: Federated Learning for Remote Sensing Scene Classification Using Vision-Language Models Hui Lin Chao Zhang Danfeng Hong Kexin Dong Congcong Wen FedML VLM 57 4 0 05 Jan 2025
Towards Multimodal Metaphor Understanding: A Chinese Dataset and Model for Metaphor Mapping Identification Dongyu Zhang Shengcheng Yin Jiahao Yu Zhiyao Wu Zhen Li Chengpei Xu Xuben Wang Feng Xia 193 0 0 05 Jan 2025
FOLDER: Accelerating Multi-modal Large Language Models with Enhanced Performance Haicheng Wang Zhemeng Yu Gabriele Spadaro Chen Ju Victor Quétu Enzo Tartaglione Enzo Tartaglione VLM 212 3 0 05 Jan 2025
MLVU: Benchmarking Multi-task Long Video Understanding Yueze Wang Yan Shu Bo Zhao Boya Wu Junjie Zhou ... Xi Yang Y. Xiong Bo Zhang Tiejun Huang Zheng Liu VLM 63 33 0 03 Jan 2025
HarmonyIQA: Pioneering Benchmark and Model for Image Harmonization Quality Assessment Zitong Xu Huiyu Duan Guangji Ma Liu Yang Jiarui Wang Qingbo Wu Xiongkuo Min Guangtao Zhai P. Callet 46 2 0 03 Jan 2025
Advancements in Visual Language Models for Remote Sensing: Datasets, Capabilities, and Enhancement Techniques Lijie Tao Han Zhang Haizhao Jing Yu Liu Kelu Yao Guoting Wei Xizhe Xue 46 0 0 03 Jan 2025
VisionLLM v2: An End-to-End Generalist Multimodal Large Language Model for Hundreds of Vision-Language Tasks Jiannan Wu Muyan Zhong Sen Xing Zeqiang Lai Zhaoyang Liu ... Lewei Lu Tong Lu Ping Luo Yu Qiao Jifeng Dai MLLM VLM LRM 104 48 0 03 Jan 2025
VLM2Vec: Training Vision-Language Models for Massive Multimodal Embedding Tasks Ziyan Jiang Rui Meng Xinyi Yang Semih Yavuz Yingbo Zhou Wenhu Chen MLLM VLM 61 21 0 03 Jan 2025
Instruction-Guided Scene Text Recognition Yongkun Du Z. Chen Yuchen Su Caiyan Jia Yu-Gang Jiang 78 3 0 03 Jan 2025
GPT4Scene: Understand 3D Scenes from Videos with Vision-Language Models Zhangyang Qi Zhixiong Zhang Ye Fang Jiaqi Wang Hengshuang Zhao 88 7 0 02 Jan 2025
2.5 Years in Class: A Multimodal Textbook for Vision-Language Pretraining Wenqi Zhang Hang Zhang Xin Li Jiashuo Sun Yongliang Shen Weiming Lu Deli Zhao Yueting Zhuang Lidong Bing VLM 56 2 0 01 Jan 2025
RORem: Training a Robust Object Remover with Human-in-the-Loop Ruibin Li Tao Yang Song Guo Lefei Zhang 66 3 0 01 Jan 2025
Generative Landmarks Guided Eyeglasses Removal 3D Face Reconstruction Dapeng Zhao Yue Qi 3DH CVBM 3DV 39 6 0 31 Dec 2024
VideoChat-Flash: Hierarchical Compression for Long-Context Video Modeling Xinhao Li Yi Wang Jiashuo Yu Xiangyu Zeng Yuhan Zhu ... Yinan He Chenting Wang Yu Qiao Yali Wang L. Wang VLM 89 26 0 31 Dec 2024
M $^3$ oralBench: A MultiModal Moral Benchmark for LVLMs Bei Yan Jie M. Zhang Zhiyuan Chen Shiguang Shan Xilin Chen ELM 54 1 0 31 Dec 2024
VideoRefer Suite: Advancing Spatial-Temporal Object Understanding with Video LLM Yuqian Yuan Hang Zhang Wentong Li Zesen Cheng Boqiang Zhang ... Deli Zhao Wenqiao Zhang Yueting Zhuang Jianke Zhu Lidong Bing 80 5 0 31 Dec 2024
A Comprehensive Survey of Large Language Models and Multimodal Large Language Models in Medicine Hanguang Xiao Feizhong Zhou Xianglong Liu Tianqi Liu Zhipeng Li Xin Liu Xiaoxuan Huang AILaw LM&MA LRM 66 19 0 31 Dec 2024
ChartAdapter: Large Vision-Language Model for Chart Summarization Peixin Xu Yujuan Ding Wenqi Fan 32 2 0 31 Dec 2024