Title
Perception Test: A Diagnostic Benchmark for Multimodal Video Models Viorica Puatruaucean Lucas Smaira Ankush Gupta Adrià Recasens Continente L. Markeeva ... Y. Aytar Simon Osindero Dima Damen Andrew Zisserman João Carreira VLM 135 140 0 23 May 2023
i-Code V2: An Autoregressive Generation Framework over Vision, Language, and Speech Data Ziyi Yang Mahmoud Khademi Yichong Xu Reid Pryzant Yuwei Fang ... Yu Shi Lu Yuan Takuya Yoshioka Michael Zeng Xuedong Huang 17 2 0 21 May 2023
On Uni-Modal Feature Learning in Supervised Multi-Modal Learning Chenzhuang Du Jiaye Teng Tingle Li Yichen Liu Tianyuan Yuan Yue Wang Yang Yuan Hang Zhao 83 38 0 02 May 2023
A Hierarchical Regression Chain Framework for Affective Vocal Burst Recognition Jinchao Li Xixin Wu Kaitao Song Dongsheng Li Xunying Liu Helen M. Meng 20 2 0 14 Mar 2023
Quantifying & Modeling Multimodal Interactions: An Information Decomposition Framework Paul Pu Liang Yun Cheng Xiang Fan Chun Kai Ling Suzanne Nie ... Nicholas B. Allen Randy P. Auerbach Faisal Mahmood Ruslan Salakhutdinov Louis-Philippe Morency 40 29 0 23 Feb 2023
MuG: A Multimodal Classification Benchmark on Game Data with Tabular, Textual, and Visual Fields Jiaying Lu Yongchen Qian Shifan Zhao Yuanzhe Xi Carl Yang VLM 24 3 0 06 Feb 2023
Multimodality Representation Learning: A Survey on Evolution, Pretraining and Its Applications Muhammad Arslan Manzoor S. Albarri Ziting Xian Zaiqiao Meng Preslav Nakov Shangsong Liang AI4TS 25 26 0 01 Feb 2023
WebUI: A Dataset for Enhancing Visual UI Understanding with Web Semantics Jason Wu Siyan Wang Siman Shen Yi-Hao Peng Jeffrey Nichols Jeffrey P. Bigham 21 68 0 30 Jan 2023
Generalizing Multimodal Variational Methods to Sets Jinzhao Zhou Yiqun Duan Zhihong Chen Yu-Cheng Chang Chin-Teng Lin DRL 45 0 0 19 Dec 2022
See, Hear, and Feel: Smart Sensory Fusion for Robotic Manipulation Hao Li Yizhi Zhang Junzhe Zhu Shaoxiong Wang Michelle A. Lee Huazhe Xu Edward H. Adelson Li Fei-Fei Ruohan Gao Jiajun Wu 32 58 0 07 Dec 2022
Multimodal Learning for Multi-Omics: A Survey Sina Tabakhi M. N. I. Suvon Pegah Ahadian Haiping Lu 15 9 0 29 Nov 2022
Foundations and Trends in Multimodal Machine Learning: Principles, Challenges, and Open Questions Paul Pu Liang Amir Zadeh Louis-Philippe Morency 18 60 0 07 Sep 2022
Progressive Fusion for Multimodal Integration Shiv Shankar Laure Thompson M. Fiterau 31 3 0 01 Sep 2022
Make Acoustic and Visual Cues Matter: CH-SIMS v2.0 Dataset and AV-Mixup Consistent Module Yih-Ling Liu Ziqi Yuan Huisheng Mao Zhiyun Liang Wanqiuyue Yang Yuanzhe Qiu Tie Cheng Xiaoteng Li Hua Xu Kai Gao 29 44 0 22 Aug 2022
Robustness Analysis of Video-Language Models Against Visual and Language Perturbations Madeline Chantry Schiappa Shruti Vyas Hamid Palangi Y. S. Rawat Vibhav Vineet VLM 120 17 0 05 Jul 2022
Self-Supervised Learning for Videos: A Survey Madeline Chantry Schiappa Y. S. Rawat M. Shah SSL 36 131 0 18 Jun 2022
Multimodal Learning with Transformers: A Survey P. Xu Xiatian Zhu David A. Clifton ViT 54 527 0 13 Jun 2022
The Modality Focusing Hypothesis: Towards Understanding Crossmodal Knowledge Distillation Zihui Xue Zhengqi Gao Sucheng Ren Hang Zhao 24 37 0 13 Jun 2022
i-Code: An Integrative and Composable Multimodal Learning Framework Ziyi Yang Yuwei Fang Chenguang Zhu Reid Pryzant Dongdong Chen ... Bin Xiao Yuanxun Lu Takuya Yoshioka Michael Zeng Xuedong Huang 40 45 0 03 May 2022
SHAPE: An Unified Approach to Evaluate the Contribution and Cooperation of Individual Modalities Pengbo Hu Xingyu Li Yi Zhou 30 10 0 30 Apr 2022
Brainish: Formalizing A Multimodal Language for Intelligence and Consciousness Paul Pu Liang 22 4 0 14 Apr 2022
Dynamic Multimodal Fusion Zihui Xue R. Marculescu 37 48 0 31 Mar 2022
Modality Competition: What Makes Joint Training of Multi-modal Network Fail in Deep Learning? (Provably) Yu Huang Junyang Lin Chang Zhou Hongxia Yang Longbo Huang 16 89 0 23 Mar 2022
DIME: Fine-grained Interpretations of Multimodal Models via Disentangled Local Explanations Yiwei Lyu Paul Pu Liang Zihao Deng Ruslan Salakhutdinov Louis-Philippe Morency 24 30 0 03 Mar 2022
High-Modality Multimodal Transformer: Quantifying Modality & Interaction Heterogeneity for High-Modality Representation Learning Paul Pu Liang Yiwei Lyu Xiang Fan Jeffrey Tsaw Yudong Liu Shentong Mo Dani Yogatama Louis-Philippe Morency Ruslan Salakhutdinov 17 29 0 02 Mar 2022
Geometric Multimodal Contrastive Representation Learning Petra Poklukar Miguel Vasco Hang Yin Francisco S. Melo Ana Paiva Danica Kragic 24 46 0 07 Feb 2022
Cross-lingual Transfer for Speech Processing using Acoustic Language Similarity Peter Wu Jiatong Shi Yifan Zhong Shinji Watanabe A. Black 19 8 0 02 Nov 2021
Multimodal Emotion-Cause Pair Extraction in Conversations Fanfan Wang Zixiang Ding Rui Xia Zhaoyu Li Jianfei Yu 10 49 0 15 Oct 2021
Zero-Shot Text-to-Image Generation Aditya A. Ramesh Mikhail Pavlov Gabriel Goh Scott Gray Chelsea Voss Alec Radford Mark Chen Ilya Sutskever VLM 255 4,781 0 24 Feb 2021
MUFASA: Multimodal Fusion Architecture Search for Electronic Health Records Zhen Xu David R. So Andrew M. Dai Mamba 58 51 0 03 Feb 2021
Removing Bias in Multi-modal Classifiers: Regularization by Maximizing Functional Entropies Itai Gat Idan Schwartz A. Schwing Tamir Hazan 55 89 0 21 Oct 2020
Supervised Multimodal Bitransformers for Classifying Images and Text Douwe Kiela Suvrat Bhooshan Hamed Firooz Ethan Perez Davide Testuggine 59 241 0 06 Sep 2019
A Survey on Bias and Fairness in Machine Learning Ninareh Mehrabi Fred Morstatter N. Saxena Kristina Lerman Aram Galstyan SyDa FaML 323 4,212 0 23 Aug 2019
GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding Alex Jinpeng Wang Amanpreet Singh Julian Michael Felix Hill Omer Levy Samuel R. Bowman ELM 297 6,959 0 20 Apr 2018
Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learning Y. Gal Zoubin Ghahramani UQCV BDL 285 9,138 0 06 Jun 2015