LLaVA-Med: Training a Large Language-and-Vision Assistant for Biomedicine in One Day

1 June 2023

Jianwei Yang

Papers citing "LLaVA-Med: Training a Large Language-and-Vision Assistant for Biomedicine in One Day"

45 / 145 papers shown

Title
DALL-M: Context-Aware Clinical Data Augmentation with LLMs Chihcheng Hsieh Catarina Moreira Isabel Blanco Nobre Sandra Costa Sousa Chun Ouyang M. Brereton Joaquim A. Jorge Jacinto C. Nascimento 54 0 0 11 Jul 2024
WSI-VQA: Interpreting Whole Slide Images by Generative Visual Question Answering Pingyi Chen Chenglu Zhu Sunyi Zheng Honglin Li Lin Yang 55 7 0 08 Jul 2024
CaRe-Ego: Contact-aware Relationship Modeling for Egocentric Interactive Hand-object Segmentation Yuejiao Su Yi Wang Lap-Pui Chau 65 1 0 08 Jul 2024
LLaRA: Supercharging Robot Learning Data for Vision-Language Policy Xiang Li Cristina Mata J. Park Kumara Kahatapitiya Yoo Sung Jang ... Kanchana Ranasinghe R. Burgert Mu Cai Yong Jae Lee Michael S. Ryoo LM&Ro 72 26 0 28 Jun 2024
Benchmarking Generative Models on Computational Thinking Tests in Elementary Visual Programming Victor-Alexandru Pădurean Adish Singla ELM 54 3 0 14 Jun 2024
Dr-LLaVA: Visual Instruction Tuning with Symbolic Clinical Grounding Shenghuan Sun Gregory M. Goldgof Alexander Schubert Zhiqing Sun Thomas Hartvigsen A. Butte Ahmed Alaa LM&MA 42 4 0 29 May 2024
Enhancing Zero-Shot Facial Expression Recognition by LLM Knowledge Transfer Zengqun Zhao Yu Cao Shaogang Gong Ioannis Patras 53 6 0 29 May 2024
MetaToken: Detecting Hallucination in Image Descriptions by Meta Classification Laura Fieback Jakob Spiegelberg Hanno Gottschalk MLLM 65 5 0 29 May 2024
A Misleading Gallery of Fluid Motion by Generative Artificial Intelligence Ali Kashefi VGen 51 5 0 24 May 2024
A Textbook Remedy for Domain Shifts: Knowledge Priors for Medical Image Analysis Yue Yang Mona Gandhi Yufei Wang Yifan Wu Michael S. Yao Christopher Callison-Burch James C. Gee Mark Yatskar 58 3 0 23 May 2024
BiomedParse: a biomedical foundation model for image parsing of everything everywhere all at once Theodore Zhao Yu Gu Jianwei Yang Naoto Usuyama Ho Hin Lee ... B. Piening Carlo Bifulco Mu-Hsin Wei Hoifung Poon Sheng Wang MedIm 36 23 0 21 May 2024
A Survey of Deep Learning-based Radiology Report Generation Using Multimodal Data Xinyi Wang Grazziela Figueredo Ruizhe Li W. Zhang Weitong Chen Xin Chen MedIm ViT 49 2 0 21 May 2024
Simplifying Multimodality: Unimodal Approach to Multimodal Challenges in Radiology with General-Domain Large Language Model Seonhee Cho Choonghan Kim Jiho Lee Chetan Chilkunda Sujin Choi Joo Heung Yoon 53 0 0 29 Apr 2024
BlenderAlchemy: Editing 3D Graphics with Vision-Language Models Ian Huang Guandao Yang Leonidas J. Guibas 34 3 0 26 Apr 2024
RadGenome-Chest CT: A Grounded Vision-Language Dataset for Chest CT Analysis Xiaoman Zhang Chaoyi Wu Ziheng Zhao Jiayu Lei Ya Zhang Yanfeng Wang Weidi Xie 28 16 0 25 Apr 2024
SkinGEN: an Explainable Dermatology Diagnosis-to-Generation Framework with Interactive Vision-Language Models Bo Lin Yingjing Xu Xuanwen Bao Zhou Zhao Zuyong Zhang Zhouyang Wang 61 2 0 23 Apr 2024
LaPA: Latent Prompt Assist Model For Medical Visual Question Answering Tiancheng Gu Kaicheng Yang Dongnan Liu Weidong Cai MedIm 41 2 0 19 Apr 2024
Tri-modal Confluence with Temporal Dynamics for Scene Graph Generation in Operating Rooms Diandian Guo Manxi Lin Jialun Pei He Tang Yueming Jin Pheng-Ann Heng 37 2 0 14 Apr 2024
Voice EHR: Introducing Multimodal Audio Data for Health James Anibal Hannah Huth Ming Li Lindsey A Hazen Y. Lam ... Emily Ricotta David A. Clifton Louise Thwaites Yael Bensoussan Bradford J. Wood 37 1 0 02 Apr 2024
Surgical-LVLM: Learning to Adapt Large Vision-Language Model for Grounded Visual Question Answering in Robotic Surgery Guan-Feng Wang Long Bai Wan Jun Nah Jie Wang Zhaoxi Zhang Zhen Chen Jinlin Wu Mobarakol Islam Hongbin Liu Hongliang Ren 46 14 0 22 Mar 2024
Beyond Text: Frozen Large Language Models in Visual Signal Comprehension Lei Zhu Fangyun Wei Yanye Lu MLLM VLM 52 17 0 12 Mar 2024
Can LLMs' Tuning Methods Work in Medical Multimodal Domain? Jiawei Chen Yue Jiang Dingkang Yang Mingcheng Li Jinjie Wei Ziyun Qian Lihua Zhang LM&MA 27 9 0 11 Mar 2024
General surgery vision transformer: A video pre-trained foundation model for general surgery Samuel Schmidgall Ji Woong Kim Jeffery Jopling Axel Krieger ViT MedIm 36 5 0 09 Mar 2024
Debiasing Multimodal Large Language Models Yi-Fan Zhang Weichen Yu Qingsong Wen Xue Wang Zhang Zhang Liang Wang Rong Jin Tien-Ping Tan 53 4 0 08 Mar 2024
Effectiveness Assessment of Recent Large Vision-Language Models Yao Jiang Xinyu Yan Ge-Peng Ji Keren Fu Meijun Sun Huan Xiong Deng-Ping Fan Fahad Shahbaz Khan 37 14 0 07 Mar 2024
CoTBal: Comprehensive Task Balancing for Multi-Task Visual Instruction Tuning Yanqi Dai Dong Jing Nanyi Fei Zhiwu Lu Nanyi Fei Guoxing Yang Zhiwu Lu 55 3 0 07 Mar 2024
Beyond Specialization: Assessing the Capabilities of MLLMs in Age and Gender Estimation Maksim Kuprashevich Grigorii Alekseenko Irina Tolstykh ELM 61 4 0 04 Mar 2024
Aligning Modalities in Vision Large Language Models via Preference Fine-tuning Yiyang Zhou Chenhang Cui Rafael Rafailov Chelsea Finn Huaxiu Yao VLM MLLM 38 89 0 18 Feb 2024
Where is the answer? Investigating Positional Bias in Language Model Knowledge Extraction Kuniaki Saito Kihyuk Sohn Chen-Yu Lee Yoshitaka Ushiku 66 2 0 16 Feb 2024
Towards Urban General Intelligence: A Review and Outlook of Urban Foundation Models Weijiao Zhang Jindong Han Zhao Xu Hang Ni Hao Liu Hui Xiong Hui Xiong AI4CE 79 15 0 30 Jan 2024
LLaVA-MoLE: Sparse Mixture of LoRA Experts for Mitigating Data Conflicts in Instruction Finetuning MLLMs Shaoxiang Chen Zequn Jie Lin Ma MoE 45 47 0 29 Jan 2024
Hallucination Benchmark in Medical Visual Question Answering Jinge Wu Yunsoo Kim Honghan Wu 25 9 0 11 Jan 2024
Jack of All Tasks, Master of Many: Designing General-purpose Coarse-to-Fine Vision-Language Model Shraman Pramanick Guangxing Han Rui Hou Sayan Nag Ser-Nam Lim Nicolas Ballas Qifan Wang Rama Chellappa Amjad Almahairi VLM MLLM 48 29 0 19 Dec 2023
Quilt-LLaVA: Visual Instruction Tuning by Extracting Localized Narratives from Open-Source Histopathology Videos M. S. Seyfioglu Wisdom O. Ikezogwo Fatemeh Ghezloo Ranjay Krishna Linda G. Shapiro 32 37 0 07 Dec 2023
Dolphins: Multimodal Language Model for Driving Yingzi Ma Yulong Cao Jiachen Sun Marco Pavone Chaowei Xiao MLLM 38 50 0 01 Dec 2023
Consensus, dissensus and synergy between clinicians and specialist foundation models in radiology report generation Ryutaro Tanno D. G. Barrett Andrew Sellergren Sumedh Ghaisas Sumanth Dathathri ... S. Shetty Pushmeet Kohli Po-Sen Huang Alan Karthikesalingam Ira Ktena MedIm 30 11 0 30 Nov 2023
To See is to Believe: Prompting GPT-4V for Better Visual Instruction Tuning Junke Wang Lingchen Meng Zejia Weng Bo He Zuxuan Wu Yu-Gang Jiang MLLM VLM 38 94 0 13 Nov 2023
Multimodal ChatGPT for Medical Applications: an Experimental Study of GPT-4V Zhiling Yan Kai Zhang Rong Zhou Lifang He Xiang Li Lichao Sun LM&MA 32 48 0 29 Oct 2023
Local Large Language Models for Complex Structured Medical Tasks V. Bumgardner Aaron D. Mullen Samuel E. Armstrong Caylin D. Hickey Jeffrey A. Talbert 36 5 0 03 Aug 2023
Med-Flamingo: a Multimodal Medical Few-shot Learner Michael Moor Qian Huang Shirley Wu Michihiro Yasunaga C. Zakka Yashodhara Dalmia E. Reis Pranav Rajpurkar J. Leskovec LM&MA MedIm 27 233 0 27 Jul 2023
CephGPT-4: An Interactive Multimodal Cephalometric Measurement and Diagnostic System with Visual Large Language Model Lei Ma Jincong Han Zhaoxin Wang Dian Zhang LM&MA 30 8 0 01 Jul 2023
OphGLM: Training an Ophthalmology Large Language-and-Vision Assistant based on Instructions and Dialogue Weihao Gao Zhuo Deng Zhiyuan Niu Fuju Rong Chucheng Chen ... Fangjun Li Zhenjie Cao Zhaoyi Ma Wenbin Wei Lan Ma LM&MA 31 33 0 21 Jun 2023
Parameter-Efficient Fine-Tuning for Medical Image Analysis: The Missed Opportunity Raman Dutt Linus Ericsson Pedro Sanchez Sotirios A. Tsaftaris Timothy M. Hospedales MedIm 35 50 0 14 May 2023
Instruction Tuning with GPT-4 Baolin Peng Chunyuan Li Pengcheng He Michel Galley Jianfeng Gao SyDa ALM LM&MA 171 579 0 06 Apr 2023
Learn to Explain: Multimodal Reasoning via Thought Chains for Science Question Answering Pan Lu Swaroop Mishra Tony Xia Liang Qiu Kai-Wei Chang Song-Chun Zhu Oyvind Tafjord Peter Clark Ashwin Kalyan ELM ReLM LRM 211 1,113 0 20 Sep 2022