Robust Multimodal Learning with Missing Modalities via Parameter-Efficient Adaptation

6 October 2023

Papers citing "Robust Multimodal Learning with Missing Modalities via Parameter-Efficient Adaptation"

41 / 41 papers shown

Title
Exploring Missing Modality in Multimodal Egocentric Datasets Merey Ramazanova Alejandro Pardo Humam Alwassel Guohao Li EgoV 62 4 0 21 Jan 2024
What Makes for Robust Multi-Modal Models in the Face of Missing Modalities? Siting Li Chenzhuang Du Yue Zhao Yu Huang Hang Zhao 36 4 0 10 Oct 2023
Delivering Arbitrary-Modal Semantic Segmentation Jiaming Zhang R. Liu Haowen Shi Kailun Yang Simon Reiß Kunyu Peng Haodong Fu Kaiwei Wang Rainer Stiefelhagen VLM 95 99 0 02 Mar 2023
Towards Good Practices for Missing Modality Robust Action Recognition Sangmin Woo Sumin Lee Yeonju Park Muhammad Adi Nugroho Changick Kim 75 50 0 25 Nov 2022
A Unified Multimodal De- and Re-coupling Framework for RGB-D Motion Recognition Benjia Zhou Pichao Wang Jun Wan Yan-Ni Liang Fan Wang 61 19 0 16 Nov 2022
MM-Align: Learning Optimal Transport-based Alignment Dynamics for Fast and Accurate Inference on Missing Modality Sequences Wei Han Hui Chen MingSung Kan Soujanya Poria 77 10 0 23 Oct 2022
Scaling & Shifting Your Features: A New Baseline for Efficient Model Tuning Dongze Lian Daquan Zhou Jiashi Feng Xinchao Wang 81 258 0 17 Oct 2022
Multimodal Learning with Transformers: A Survey Peng Xu Xiatian Zhu David Clifton ViT 156 566 0 13 Jun 2022
Analyzing Modality Robustness in Multimodal Sentiment Analysis Devamanyu Hazarika Yingting Li Bo Cheng Shuai Zhao Roger Zimmermann Soujanya Poria 80 33 0 30 May 2022
Multimodal Token Fusion for Vision Transformers Yikai Wang Xinghao Chen Lele Cao Wen-bing Huang Gang Hua Yunhe Wang ViT 84 179 0 19 Apr 2022
Are Multimodal Transformers Robust to Missing Modality? Mengmeng Ma Jian Ren Long Zhao Davide Testuggine Xi Peng ViT 93 154 0 12 Apr 2022
MultiMAE: Multi-modal Multi-task Masked Autoencoders Roman Bachmann David Mizrahi Andrei Atanov Amir Zamir 127 276 0 04 Apr 2022
Decoupling and Recoupling Spatiotemporal Representation for RGB-D-based Motion Recognition Benjia Zhou Pichao Wang Jun Wan Yanyan Liang Fan Wang Du Zhang Zhen Lei Hao Li Rong Jin 69 30 0 16 Dec 2021
Masked Autoencoders Are Scalable Vision Learners Kaiming He Xinlei Chen Saining Xie Yanghao Li Piotr Dollár Ross B. Girshick ViT TPM 460 7,757 0 11 Nov 2021
Towards a Unified View of Parameter-Efficient Transfer Learning Junxian He Chunting Zhou Xuezhe Ma Taylor Berg-Kirkpatrick Graham Neubig AAML 129 937 0 08 Oct 2021
Robust fine-tuning of zero-shot models Mitchell Wortsman Gabriel Ilharco Jong Wook Kim Mike Li Simon Kornblith ... Raphael Gontijo-Lopes Hannaneh Hajishirzi Ali Farhadi Hongseok Namkoong Ludwig Schmidt VLM 129 725 0 04 Sep 2021
Learning Deep Multimodal Feature Representation with Asymmetric Multi-layer Fusion Yikai Wang F. Sun Ming Lu Anbang Yao 74 48 0 11 Aug 2021
BitFit: Simple Parameter-efficient Fine-tuning for Transformer-based Masked Language-models Elad Ben-Zaken Shauli Ravfogel Yoav Goldberg 165 1,223 0 18 Jun 2021
SegFormer: Simple and Efficient Design for Semantic Segmentation with Transformers Enze Xie Wenhai Wang Zhiding Yu Anima Anandkumar J. Álvarez Ping Luo ViT 282 5,030 0 31 May 2021
Swin Transformer: Hierarchical Vision Transformer using Shifted Windows Ze Liu Yutong Lin Yue Cao Han Hu Yixuan Wei Zheng Zhang Stephen Lin B. Guo ViT 447 21,418 0 25 Mar 2021
SMIL: Multimodal Learning with Severely Missing Modality Mengmeng Ma Jian Ren Long Zhao Sergey Tulyakov Cathy H. Wu Xi Peng 94 262 0 09 Mar 2021
Learning Modality-Specific Representations with Self-Supervised Multi-Task Learning for Multimodal Sentiment Analysis Wenmeng Yu Hua Xu Ziqi Yuan Jiele Wu SSL 90 464 0 09 Feb 2021
ViLT: Vision-and-Language Transformer Without Convolution or Region Supervision Wonjae Kim Bokyung Son Ildoo Kim VLM CLIP 119 1,745 0 05 Feb 2021
Bi-directional Cross-Modality Feature Propagation with Separation-and-Aggregation Gate for RGB-D Semantic Segmentation Xiaokang Chen Kwan-Yee Lin Jingbo Wang Wayne Wu Chao Qian Hongsheng Li Gang Zeng MDE 84 314 0 17 Jul 2020
Hetero-Modal Variational Encoder-Decoder for Joint Modality Completion and Segmentation Reuben Dorent Samuel Joutard Marc Modat Sébastien Ourselin Tom Vercauteren MedIm 56 123 0 25 Jul 2019
Multimodal End-to-End Autonomous Driving Yi Xiao Felipe Codevilla A. Gurram O. Urfalioglu Antonio M. López 54 244 0 07 Jun 2019
Multimodal Transformer with Multi-View Visual Representation for Image Captioning Jun-chen Yu Jing Li Zhou Yu Qingming Huang ViT 61 384 0 20 May 2019
Missing MRI Pulse Sequence Synthesis using Multi-Modal Generative Adversarial Network Anmol Sharma Ghassan Hamarneh MedIm GAN 137 176 0 27 Apr 2019
EmbraceNet: A robust deep learning architecture for multimodal classification Jun-Ho Choi Jong-Seok Lee 51 125 0 19 Apr 2019
Modality Distillation with Multiple Stream Networks for Action Recognition Nuno C. Garcia Pietro Morerio Vittorio Murino 65 183 0 19 Jun 2018
Group Normalization Yuxin Wu Kaiming He 228 3,660 0 22 Mar 2018
Graph Distillation for Action Detection with Privileged Modalities Zelun Luo Jun-Ting Hsieh Lu Jiang Juan Carlos Niebles Li Fei-Fei 75 104 0 30 Nov 2017
Tensor Fusion Network for Multimodal Sentiment Analysis Amir Zadeh Minghai Chen Soujanya Poria Min Zhang Louis-Philippe Morency 77 1,236 0 23 Jul 2017
Multimodal Machine Learning: A Survey and Taxonomy T. Baltrušaitis Chaitanya Ahuja Louis-Philippe Morency 101 2,932 0 26 May 2017
Layer Normalization Jimmy Lei Ba J. Kiros Geoffrey E. Hinton 413 10,494 0 21 Jul 2016
Fully Convolutional Networks for Semantic Segmentation Evan Shelhamer Jonathan Long Trevor Darrell VOS SSeg 741 37,862 0 20 May 2016
Multi-Scale Context Aggregation by Dilated Convolutions Feng Yu V. Koltun SSeg 268 8,446 0 23 Nov 2015
Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift Sergey Ioffe Christian Szegedy OOD 463 43,305 0 11 Feb 2015
ModDrop: adaptive multi-modal gesture recognition Natalia Neverova Christian Wolf Graham W. Taylor Florian Nebout 95 295 0 31 Dec 2014
Learning Rich Features from RGB-D Images for Object Detection and Segmentation Saurabh Gupta Ross B. Girshick Pablo Arbeláez Jitendra Malik ObjD 122 1,561 0 22 Jul 2014
Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translation Kyunghyun Cho B. V. Merrienboer Çağlar Gülçehre Dzmitry Bahdanau Fethi Bougares Holger Schwenk Yoshua Bengio AIMat 1.0K 23,354 0 03 Jun 2014