Learning Multi-modal Representations by Watching Hundreds of Surgical Video Lectures

27 July 2023

Nicolas Padoy

Papers citing "Learning Multi-modal Representations by Watching Hundreds of Surgical Video Lectures"

27 / 27 papers shown

Title
Multimodal Graph Representation Learning for Robust Surgical Workflow Recognition with Adversarial Feature Disentanglement Long Bai Boyi Ma Ruohan Wang Guankun Wang Beilei Cui ... Mobarakol Islam Zhe Min Jiewen Lai Nassir Navab Hongliang Ren 46 0 0 03 May 2025
Multimodal Large Language Models for Medicine: A Comprehensive Survey Jiarui Ye Hao Tang LM&MA 89 0 0 29 Apr 2025
Systematic Evaluation of Large Vision-Language Models for Surgical Artificial Intelligence Anita Rau Mark Endo Josiah Aklilu Jaewoo Heo Khaled Saab Alberto Paderno Jeffrey Jopling F. C. Holsinger Serena Yeung-Levy 44 0 0 03 Apr 2025
fine-CLIP: Enhancing Zero-Shot Fine-Grained Surgical Action Recognition with Vision-Language Models Saurav Sharma Didier Mutter N. Padoy VLM MedIm 42 0 0 25 Mar 2025
Watch and Learn: Leveraging Expert Knowledge and Language for Surgical Video Understanding David Gastager Ghazal Ghazaei Constantin Patsch 58 0 0 14 Mar 2025
SurgRAW: Multi-Agent Workflow with Chain-of-Thought Reasoning for Surgical Intelligence Chang Han Low Ziyue Wang Tianyi Zhang Zhitao Zeng Zhu Zhuo E. Mazomenos Yueming Jin LRM 46 1 0 13 Mar 2025
MM-OR: A Large Multimodal Operating Room Dataset for Semantic Understanding of High-Intensity Surgical Environments Ege Ozsoy Chantal Pellegrini Tobias Czempiel Felix Tristram Kun Yuan D. Bani-Harouni U. Eck Benjamin Busam Matthias Keicher Nassir Navab 76 1 0 04 Mar 2025
Recognize Any Surgical Object: Unleashing the Power of Weakly-Supervised Data Jiajie Li Brian R Quaranto Chenhui Xu Ishan Mishra Ruiyang Qin Dancheng Liu Peter C W Kim Jinjun Xiong 88 0 0 25 Jan 2025
OphCLIP: Hierarchical Retrieval-Augmented Learning for Ophthalmic Surgical Video-Language Pretraining Ming Hu Kun Yuan Yaling Shen Feilong Tang Xiaohao Xu ... Jin Ye N. Padoy Nassir Navab Junjun He Zongyuan Ge VLM CLIP 91 11 0 23 Nov 2024
Procedure-Aware Surgical Video-language Pretraining with Hierarchical Knowledge Augmentation Kun Yuan V. Srivastav Nassir Navab N. Padoy 44 7 0 30 Sep 2024
$VidLPRO: A $\underline{Vid}$eo-$\underline{L}$anguage $\underline{P}$re-training Framework for $\underline{Ro}$botic and Laparoscopic Surgery$ VidLPRO: A $\underline{Vid}$ eo- $\underline{L}$ anguage $\underline{P}$ re-training Framework for $\underline{Ro}$ botic and Laparoscopic Surgery Mohammadmahdi Honarmand Muhammad Abdullah Jamal Omid Mohareri 58 1 0 07 Sep 2024
GP-VLS: A general-purpose vision language model for surgery Samuel Schmidgall Joseph Cho C. Zakka W. Hiesinger LM&MA 44 5 0 27 Jul 2024
OphNet: A Large-Scale Video Benchmark for Ophthalmic Surgical Workflow Understanding Ming Hu Peng Xia Lin Wang Siyuan Yan Feilong Tang ... Xuelian Cheng Jun Cheng Chi Liu Kaijing Zhou Zongyuan Ge 35 17 0 11 Jun 2024
HecVL: Hierarchical Video-Language Pretraining for Zero-shot Surgical Phase Recognition Kun Yuan V. Srivastav Nassir Navab N. Padoy 41 11 0 16 May 2024
Enhancing Gait Video Analysis in Neurodegenerative Diseases by Knowledge Augmentation in Vision Language Model Diwei Wang Kun Yuan Candice Müller Frédéric Blanc N. Padoy Hyewon Seo 39 2 0 20 Mar 2024
Medical Vision Language Pretraining: A survey Prashant Shrestha Sanskar Amgain Bidur Khanal Cristian A. Linte Binod Bhattarai VLM 32 14 0 11 Dec 2023
Text-Only Training for Image Captioning using Noise-Injected CLIP David Nukrai Ron Mokady Amir Globerson VLM CLIP 52 94 0 01 Nov 2022
Multi-Modal Masked Autoencoders for Medical Vision-and-Language Pre-Training Zhihong Chen Yu Du Jinpeng Hu Yang Liu Guanbin Li Xiang Wan Tsung-Hui Chang 86 111 0 15 Sep 2022
BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generation Junnan Li Dongxu Li Caiming Xiong S. Hoi MLLM BDL VLM CLIP 390 4,125 0 28 Jan 2022
Ego4D: Around the World in 3,000 Hours of Egocentric Video Kristen Grauman Andrew Westbury Eugene Byrne Zachary Chavis Antonino Furnari ... Mike Zheng Shou Antonio Torralba Lorenzo Torresani Mingfei Yan Jitendra Malik EgoV 224 1,018 0 13 Oct 2021
VideoCLIP: Contrastive Pre-training for Zero-shot Video-Text Understanding Hu Xu Gargi Ghosh Po-Yao (Bernie) Huang Dmytro Okhonko Armen Aghajanyan Florian Metze Luke Zettlemoyer Florian Metze Luke Zettlemoyer Christoph Feichtenhofer CLIP VLM 259 558 0 28 Sep 2021
Conceptual 12M: Pushing Web-Scale Image-Text Pre-Training To Recognize Long-Tail Visual Concepts Soravit Changpinyo P. Sharma Nan Ding Radu Soricut VLM 275 1,081 0 17 Feb 2021
Scaling Up Visual and Vision-Language Representation Learning With Noisy Text Supervision Chao Jia Yinfei Yang Ye Xia Yi-Ting Chen Zarana Parekh Hieu H. Pham Quoc V. Le Yun-hsuan Sung Zhen Li Tom Duerig VLM CLIP 298 3,693 0 11 Feb 2021
Multi-modal Transformer for Video Retrieval Valentin Gabeur Chen Sun Alahari Karteek Cordelia Schmid ViT 412 595 0 21 Jul 2020
Improved Baselines with Momentum Contrastive Learning Xinlei Chen Haoqi Fan Ross B. Girshick Kaiming He SSL 267 3,369 0 09 Mar 2020
EndoNet: A Deep Architecture for Recognition Tasks on Laparoscopic Videos A. P. Twinanda S. Shehata Didier Mutter J. Marescaux M. de Mathelin N. Padoy 173 840 0 09 Feb 2016
Efficient Estimation of Word Representations in Vector Space Tomáš Mikolov Kai Chen G. Corrado J. Dean 3DV 233 31,253 0 16 Jan 2013