Visual Instruction Tuning

17 April 2023

Papers citing "Visual Instruction Tuning"

50 / 3,324 papers shown

Title
Med-2E3: A 2D-Enhanced 3D Medical Multimodal Large Language Model Yiming Shi Xun Zhu Ying Hu Chenyi Guo Miao Li Ji Wu 91 1 0 19 Nov 2024
Generative Timelines for Instructed Visual Assembly Alejandro Pardo Jui-hsien Wang Guohao Li Josef Sivic Bryan C. Russell Fabian Caba Heilbron VGen 76 0 0 19 Nov 2024
VILA-M3: Enhancing Vision-Language Models with Medical Expert Knowledge Vishwesh Nath Wenqi Li Dong Yang Andriy Myronenko Mingxin Zheng ... Holger Roth Daguang Xu Baris Turkbey Holger Roth Daguang Xu VLM 123 4 0 19 Nov 2024
TrojanRobot: Physical-World Backdoor Attacks Against VLM-based Robotic Manipulation Xiaobei Wang Hewen Pan Hangtao Zhang Minghui Li Shengshan Hu ... Peijin Guo Yichen Wang Wei Wan Aishan Liu L. Zhang AAML 111 8 0 18 Nov 2024
VLN-Game: Vision-Language Equilibrium Search for Zero-Shot Semantic Navigation Bangguo Yu Yuzhen Liu Lei Han Hamidreza Kasaei Tingguang Li M. Cao LM&Ro 103 3 0 18 Nov 2024
PSA-VLM: Enhancing Vision-Language Model Safety through Progressive Concept-Bottleneck-Driven Alignment Zhendong Liu Yuanbi Nie Yingshui Tan Xiangyu Yue Qiushi Cui Chongjun Wang Xiaoyong Zhu Jian Xu Bo Zheng 85 0 0 18 Nov 2024
SignEye: Traffic Sign Interpretation from Vehicle First-Person View Chuang Yang Xu Han T. Han Yuejiao Su Junyu Gao Hongyuan Zhang Yi Wang Lap-Pui Chau 96 2 0 18 Nov 2024
AtomThink: A Slow Thinking Framework for Multimodal Mathematical Reasoning Kun Xiang Zhili Liu Zihao Jiang Yunshuang Nie Runhui Huang ... Yihan Zeng Jiawei Han Lanqing Hong Hang Xu Xiaodan Liang LRM 130 13 0 18 Nov 2024
GLDesigner: Leveraging Multi-Modal LLMs as Designer for Enhanced Aesthetic Text Glyph Layouts Junwen He Yifan Wang Lijun Wang Huchuan Lu Jun-Yan He Chong Li Hanyuan Chen Jin-Peng Lan Bin Luo Yifeng Geng 79 1 0 18 Nov 2024
Exploring the Adversarial Vulnerabilities of Vision-Language-Action Models in Robotics Taowen Wang Dongfang Liu James Liang Wenhao Yang Qifan Wang Cheng Han Jiebo Luo Ruixiang Tang Ruixiang Tang AAML 103 5 0 18 Nov 2024
MC-LLaVA: Multi-Concept Personalized Vision-Language Model Ruichuan An Sihan Yang Ming Lu Kai Zeng Yulin Luo ... Hao Liang Qi She Shanghang Zhang Wentao Zhang Wentao Zhang 107 7 0 18 Nov 2024
SoK: Unifying Cybersecurity and Cybersafety of Multimodal Foundation Models with an Information Theory Approach Ruoxi Sun Jiamin Chang Hammond Pearce Chaowei Xiao B. Li Qi Wu Surya Nepal Minhui Xue 60 0 0 17 Nov 2024
TS-LLaVA: Constructing Visual Tokens through Thumbnail-and-Sampling for Training-Free Video Large Language Models Tingyu Qu Mingxiao Li Tinne Tuytelaars Marie-Francine Moens VLM 56 2 0 17 Nov 2024
Understanding Multimodal LLMs: the Mechanistic Interpretability of Llava in Visual Question Answering Zeping Yu Sophia Ananiadou 293 0 0 17 Nov 2024
Memory-Augmented Multimodal LLMs for Surgical VQA via Self-Contained Inquiry Wenjun Hou Yi Cheng Kaishuai Xu Yan Hu Wenjie Li Jiang-Dong Liu 50 0 0 17 Nov 2024
MolParser: End-to-end Visual Recognition of Molecule Structures in the Wild Xi Fang Jiankun Wang X. Cai Shangqian Chen Shuwen Yang Lin Yao Linfeng Zhang Guolin Ke Linfeng Zhang Guolin Ke 62 1 0 17 Nov 2024
Generating Compositional Scenes via Text-to-image RGBA Instance Generation Alessandro Fontanella Petru-Daniel Tudosiu Yongxin Yang Shifeng Zhang Sarah Parisot 51 2 0 16 Nov 2024
MpoxVLM: A Vision-Language Model for Diagnosing Skin Lesions from Mpox Virus Infection Xu Cao Wenqian Ye K. Moise Megan Coffee 62 2 0 16 Nov 2024
MetaLA: Unified Optimal Linear Approximation to Softmax Attention Map Yuhong Chou Man Yao Kexin Wang Yuqi Pan Ruijie Zhu Yiran Zhong Yu Qiao Jian Wu Bo Xu Guoqi Li 54 4 0 16 Nov 2024
GeoGround: A Unified Large Vision-Language Model for Remote Sensing Visual Grounding Yimiao Zhou Mengcheng Lan Xiang Li Yiping Ke Yiping Ke Xue Jiang Qingyun Li Xue Yang Wayne Zhang ObjD VLM 132 6 0 16 Nov 2024
Thinking Before Looking: Improving Multimodal LLM Reasoning via Mitigating Visual Hallucination Haojie Zheng Tianyang Xu Hanchi Sun Shu Pu Ruoxi Chen Lichao Sun MLLM LRM 97 8 0 15 Nov 2024
VeriGraph: Scene Graphs for Execution Verifiable Robot Planning Daniel Ekpo Mara Levy Saksham Suri Chuong Huynh Abhinav Shrivastava 66 2 0 15 Nov 2024
Llama Guard 3 Vision: Safeguarding Human-AI Image Understanding Conversations Jianfeng Chi Ujjwal Karn Hongyuan Zhan Eric Michael Smith Javier Rando Yiming Zhang Kate Plawiak Zacharie Delpierre Coudert Kartikeya Upasani Mahesh Pasupuleti MLLM 3DH 67 25 0 15 Nov 2024
The Dawn of GUI Agent: A Preliminary Case Study with Claude 3.5 Computer Use Siyuan Hu Mingyu Ouyang Difei Gao Mike Zheng Shou LM&Ro LLMAG 39 17 0 15 Nov 2024
Visual-Linguistic Agent: Towards Collaborative Contextual Object Reasoning Jingru Yang Huan Yu Yang Jingxin C. Xu Yin Biao Yu Sun Shengfeng He 31 0 0 15 Nov 2024
SEAGULL: No-reference Image Quality Assessment for Regions of Interest via Vision-Language Instruction Tuning Zhaoyu Chen Juan Wang Wen Wang Sunhan Xu Hang Xiong ... Jian Guo Shuxun Wang Chun Yuan Bing Li Weiming Hu VLM 55 2 0 15 Nov 2024
AMXFP4: Taming Activation Outliers with Asymmetric Microscaling Floating-Point for 4-bit LLM Inference Janghwan Lee Jiwoong Park Jinseok Kim Yongjik Kim Jungju Oh Jinwook Oh Jungwook Choi 51 2 0 15 Nov 2024
Enhancing the Reasoning Ability of Multimodal Large Language Models via Mixed Preference Optimization Weiyun Wang Zhe Chen Wenhai Wang Yue Cao Yangzhou Liu ... Jinguo Zhu X. Zhu Lewei Lu Yu Qiao Jifeng Dai LRM 76 56 1 15 Nov 2024
Motion-Grounded Video Reasoning: Understanding and Perceiving Motion at Pixel Level Andong Deng Tongjia Chen Shoubin Yu Taojiannan Yang Lincoln Spencer Yapeng Tian Ajmal Mian Joey Tianyi Zhou Chen Chen LRM 68 2 0 15 Nov 2024
Advancing Fine-Grained Visual Understanding with Multi-Scale Alignment in Multi-Modal Models Wei Wang Zechao Li Qi Xu Linfeng Li Yiqing Cai Botian Jiang Hang Song Xingcan Hu Pengyu Wang Li Xiao 39 4 0 14 Nov 2024
Jailbreak Attacks and Defenses against Multimodal Generative Models: A Survey Xuannan Liu Xing Cui Peipei Li Zekun Li Huaibo Huang Shuhan Xia Miaoxuan Zhang Yueying Zou Ran He AAML 67 8 0 14 Nov 2024
Harnessing Vision Foundation Models for High-Performance, Training-Free Open Vocabulary Segmentation Yuheng Shi Minjing Dong Chang Xu VLM 74 1 0 14 Nov 2024
VCBench: A Controllable Benchmark for Symbolic and Abstract Challenges in Video Cognition Chenglin Li Qianglong Chen Zhi Li Feng Tao Yin Zhang 64 0 0 14 Nov 2024
Spider: Any-to-Many Multimodal LLM Jinxiang Lai Jie Zhang Jun Liu Jian Li Xiaocheng Lu Song Guo MLLM 84 2 0 14 Nov 2024
ClevrSkills: Compositional Language and Visual Reasoning in Robotics Sanjay Haresh Daniel Dijkman Apratim Bhattacharyya Roland Memisevic CoGe LRM 50 1 0 13 Nov 2024
Multimodal Instruction Tuning with Hybrid State Space Models Jianing Zhou Han Li Shuai Zhang Ning Xie Ruijie Wang Xiaohan Nie Sheng Liu Lingyun Wang 51 0 0 13 Nov 2024
NavAgent: Multi-scale Urban Street View Fusion For UAV Embodied Vision-and-Language Navigation Youzhi Liu Fanglong Yao Yuanchang Yue Guangluan Xu Xian Sun Kun Fu LM&Ro 62 3 0 13 Nov 2024
Open-World Task and Motion Planning via Vision-Language Model Inferred Constraints Nishanth Kumar F. Ramos Dieter Fox Caelan Reed Garrett Tomás Lozano-Pérez Leslie Pack Kaelbling Caelan Reed Garrett LRM LM&Ro 68 4 0 13 Nov 2024
Public Health Advocacy Dataset: A Dataset of Tobacco Usage Videos from Social Media N. V. R. Chappa Charlotte McCormick Susana Rodriguez Gongora P. Dobbs Khoa Luu 95 2 0 12 Nov 2024
Artificial Intelligence for Biomedical Video Generation Linyuan Li Jianing Qiu Anujit Saha Lin Li Poyuan Li Mengxian He Ziyu Guo Wu Yuan VGen 73 1 0 12 Nov 2024
SparrowVQE: Visual Question Explanation for Course Content Understanding Jialu Li Manish Kumar Thota Ruslan Gokhman Radek Holik Youshan Zhang 55 1 0 12 Nov 2024
SAMPart3D: Segment Any Part in 3D Objects Yanting Yang Yukun Huang Yu Guo Liangjun Lu Xiaoyang Wu Edmund Y. Lam Yan-Pei Cao Xihui Liu VLM 52 7 0 11 Nov 2024
CapeLLM: Support-Free Category-Agnostic Pose Estimation with Multimodal Large Language Models Junho Kim Hyungjin Chung Byung-Hoon Kim VLM 61 0 0 11 Nov 2024
Learning from Feedback: Semantic Enhancement for Object SLAM Using Foundation Models Jungseok Hong Ran Choi John Leonard VLM 71 1 0 11 Nov 2024
EVQAScore: A Fine-grained Metric for Video Question Answering Data Quality Evaluation Hao Liang Zirong Chen Wentao Zhang Wentao Zhang 49 1 0 11 Nov 2024
Golden Touchstone: A Comprehensive Bilingual Benchmark for Evaluating Financial Large Language Models Xiaojun Wu Junxi Liu Huanyi Su Zhouchi Lin Yiyan Qi ... Fuwei Wang Saizhuo Wang Fengrui Hua Jia Li Jian Guo 57 0 0 09 Nov 2024
M-Longdoc: A Benchmark For Multimodal Super-Long Document Understanding And A Retrieval-Aware Tuning Framework Yew Ken Chia Liying Cheng Hou Pong Chan Chaoqun Liu Maojia Song Sharifah Mahani Aljunied Soujanya Poria Lidong Bing RALM VLM 55 5 0 09 Nov 2024
Aligned Vector Quantization for Edge-Cloud Collabrative Vision-Language Models Xiao Liu Lijun Zhang Deepak Ganesan Hui Guan VLM 43 0 0 08 Nov 2024
Towards Multi-Modal Mastery: A 4.5B Parameter Truly Multi-Modal Small Language Model Ben Koska Mojmír Horváth MoE 47 1 0 08 Nov 2024
Autoregressive Models in Vision: A Survey Jing Xiong Gongye Liu Lun Huang Chengyue Wu Taiqiang Wu ... Hao Fei Guillermo Sapiro Jiebo Luo Ping Luo Ngai Wong VGen 60 9 0 08 Nov 2024