MERBench: A Unified Evaluation Benchmark for Multimodal Emotion Recognition

7 January 2024

Zheng Lian

Guoying Zhao

Yong Ren

Hao Gu

Papers citing "MERBench: A Unified Evaluation Benchmark for Multimodal Emotion Recognition"

50 / 68 papers shown

Title
Enhancing Multimodal Sentiment Analysis for Missing Modality through Self-Distillation and Unified Modality Cross-Attention Yuzhe Weng Haotian Wang Tian Gao Kewei Li Shutong Niu Jun Du 56 0 0 19 Oct 2024
Baichuan 2: Open Large-scale Language Models Ai Ming Yang Bin Xiao Bingning Wang Borong Zhang Ce Bian ... Youxin Jiang Yuchen Gao Yupeng Zhang Guosheng Dong Zhiying Wu ELM LRM 140 731 0 19 Sep 2023
Llama 2: Open Foundation and Fine-Tuned Chat Models Hugo Touvron Louis Martin Kevin R. Stone Peter Albert Amjad Almahairi ... Sharan Narang Aurelien Rodriguez Robert Stojnic Sergey Edunov Thomas Scialom AI4MH ALM 211 11,636 0 18 Jul 2023
MAE-DFER: Efficient Masked Autoencoder for Self-supervised Dynamic Facial Expression Recognition Guoying Zhao Zheng Lian B. Liu Jianhua Tao 55 17 0 05 Jul 2023
Explainable Multimodal Emotion Recognition Zheng Lian Haiyang Sun Guoying Zhao Hao Gu Zhuofan Wen ... Shan Liang Ya Li Jiangyan Yi B. Liu Jianhua Tao MLLM 40 6 0 27 Jun 2023
The RefinedWeb Dataset for Falcon LLM: Outperforming Curated Corpora with Web Data, and Web Data Only Guilherme Penedo Quentin Malartic Daniel Hesslow Ruxandra-Aimée Cojocaru Alessandro Cappelli Hamza Alobeidli B. Pannier Ebtesam Almazrouei Julien Launay 88 758 0 01 Jun 2023
DINOv2: Learning Robust Visual Features without Supervision Maxime Oquab Timothée Darcet Théo Moutakanni Huy Q. Vo Marc Szafraniec ... Hervé Jégou Julien Mairal Patrick Labatut Armand Joulin Piotr Bojanowski VLM CLIP SSL 256 3,205 0 14 Apr 2023
EVA-02: A Visual Representation for Neon Genesis Yuxin Fang Quan-Sen Sun Xinggang Wang Tiejun Huang Xinlong Wang Yue Cao VLM ViT CLIP 83 274 0 20 Mar 2023
LLaMA: Open and Efficient Foundation Language Models Hugo Touvron Thibaut Lavril Gautier Izacard Xavier Martinet Marie-Anne Lachaux ... Faisal Azhar Aurelien Rodriguez Armand Joulin Edouard Grave Guillaume Lample ALM PILM 871 12,840 0 27 Feb 2023
Robust Speech Recognition via Large-Scale Weak Supervision Alec Radford Jong Wook Kim Tao Xu Greg Brockman C. McLeavey Ilya Sutskever OffRL 113 3,515 0 06 Dec 2022
LERT: A Linguistically-motivated Pre-trained Language Model Yiming Cui Wanxiang Che Shijin Wang Ting Liu 43 25 0 10 Nov 2022
BLOOM: A 176B-Parameter Open-Access Multilingual Language Model BigScience Workshop : Teven Le Scao Angela Fan Christopher Akiki ... Zhongli Xie Zifan Ye M. Bras Younes Belkada Thomas Wolf VLM 287 2,364 0 09 Nov 2022
Make Acoustic and Visual Cues Matter: CH-SIMS v2.0 Dataset and AV-Mixup Consistent Module Yih-Ling Liu Ziqi Yuan Huisheng Mao Zhiyun Liang Wanqiuyue Yang Yuanzhe Qiu Tie Cheng Xiaoteng Li Hua Xu Kai Gao 52 45 0 22 Aug 2022
Self-Supervised Learning for Videos: A Survey Madeline Chantry Schiappa Yogesh S Rawat M. Shah SSL 79 134 0 18 Jun 2022
Self-Supervised Speech Representation Learning: A Review Abdel-rahman Mohamed Hung-yi Lee Lasse Borgholt Jakob Drachmann Havtorn Joakim Edin ... Shang-Wen Li Karen Livescu Lars Maaløe Tara N. Sainath Shinji Watanabe SSL AI4TS 159 355 0 21 May 2022
M3ED: Multi-modal Multi-scene Multi-label Emotional Dialogue Database Jinming Zhao Tenggan Zhang Jingwen Hu Yuchen Liu Qin Jin Xinchao Wang Haizhou Li 41 55 0 09 May 2022
OPT: Open Pre-trained Transformer Language Models Susan Zhang Stephen Roller Naman Goyal Mikel Artetxe Moya Chen ... Daniel Simig Punit Singh Koura Anjali Sridhar Tianlu Wang Luke Zettlemoyer VLM OSLM AI4CE 284 3,583 0 02 May 2022
UNICON: Combating Label Noise Through Uniform Selection and Contrastive Learning Nazmul Karim Mamshad Nayeem Rizve Nazanin Rahnavard Ajmal Mian M. Shah NoLa 49 98 0 28 Mar 2022
VideoMAE: Masked Autoencoders are Data-Efficient Learners for Self-Supervised Video Pre-Training Zhan Tong Yibing Song Jue Wang Limin Wang ViT 195 1,156 0 23 Mar 2022
PERT: Pre-training BERT with Permuted Language Model Yiming Cui Ziqing Yang Ting Liu 46 37 0 14 Mar 2022
GCNet: Graph Completion Network for Incomplete Multimodal Learning in Conversation Zheng Lian Lang Chen Guoying Zhao B. Liu J. Tao 54 96 0 04 Mar 2022
data2vec: A General Framework for Self-supervised Learning in Speech, Vision and Language Alexei Baevski Wei-Ning Hsu Qiantong Xu Arun Babu Jiatao Gu Michael Auli SSL VLM ViT 77 845 0 07 Feb 2022
WavLM: Large-Scale Self-Supervised Pre-Training for Full Stack Speech Processing Sanyuan Chen Chengyi Wang Zhengyang Chen Yu-Huan Wu Shujie Liu ... Yao Qian Jian Wu Micheal Zeng Xiangzhan Yu Furu Wei SSL 178 1,794 0 26 Oct 2021
Improving Multimodal Fusion with Hierarchical Mutual Information Maximization for Multimodal Sentiment Analysis Wei Han Hui Chen Soujanya Poria 47 330 0 01 Sep 2021
HuBERT: Self-Supervised Speech Representation Learning by Masked Prediction of Hidden Units Wei-Ning Hsu Benjamin Bolte Yao-Hung Hubert Tsai Kushal Lakhotia Ruslan Salakhutdinov Abdel-rahman Mohamed SSL 129 2,879 0 14 Jun 2021
SUPERB: Speech processing Universal PERformance Benchmark Shu-Wen Yang Po-Han Chi Yung-Sung Chuang Cheng-I Jeff Lai Kushal Lakhotia ... Shuyan Dong Shang-Wen Li Shinji Watanabe Abdel-rahman Mohamed Hung-yi Lee SSL 80 910 0 03 May 2021
GLM: General Language Model Pretraining with Autoregressive Blank Infilling Zhengxiao Du Yujie Qian Xiao Liu Ming Ding J. Qiu Zhilin Yang Jie Tang BDL AI4CE 93 1,520 0 18 Mar 2021
SMIL: Multimodal Learning with Severely Missing Modality Mengmeng Ma Jian Ren Long Zhao Sergey Tulyakov Cathy H. Wu Xi Peng 86 249 0 09 Mar 2021
Learning Transferable Visual Models From Natural Language Supervision Alec Radford Jong Wook Kim Chris Hallacy Aditya A. Ramesh Gabriel Goh ... Amanda Askell Pamela Mishkin Jack Clark Gretchen Krueger Ilya Sutskever CLIP VLM 696 28,659 0 26 Feb 2021
WeNet: Production oriented Streaming and Non-streaming End-to-End Speech Recognition Toolkit Zhuoyuan Yao Di Wu Xiong Wang Binbin Zhang Fan Yu Chao Yang Zhendong Peng Xiaoyu Chen Lei Xie X. Lei 49 265 0 02 Feb 2021
Deep Partial Multi-View Learning Changqing Zhang Yajie Cui Zongbo Han Qiufeng Wang Huazhu Fu Q. Hu 75 223 0 12 Nov 2020
Training Strategies to Handle Missing Modalities for Audio-Visual Expression Recognition Srinivas Parthasarathy Shiva Sundaram 44 76 0 02 Oct 2020
wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representations Alexei Baevski Henry Zhou Abdel-rahman Mohamed Michael Auli SSL 187 5,734 0 20 Jun 2020
DeBERTa: Decoding-enhanced BERT with Disentangled Attention Pengcheng He Xiaodong Liu Jianfeng Gao Weizhu Chen AAML 107 2,682 0 05 Jun 2020
MISA: Modality-Invariant and -Specific Representations for Multimodal Sentiment Analysis Devamanyu Hazarika Roger Zimmermann Soujanya Poria 52 694 0 07 May 2020
Revisiting Pre-Trained Models for Chinese Natural Language Processing Yiming Cui Wanxiang Che Ting Liu Bing Qin Shijin Wang Guoping Hu 57 693 0 29 Apr 2020
Pre-trained Models for Natural Language Processing: A Survey Xipeng Qiu Tianxiang Sun Yige Xu Yunfan Shao Ning Dai Xuanjing Huang LM&MA VLM 316 1,471 0 18 Mar 2020
Analysing Affective Behavior in the First ABAW 2020 Competition D. Kollias Attila Schulc Elnar Hajiyev Stefanos Zafeiriou CVBM 73 281 0 30 Jan 2020
Modality to Modality Translation: An Adversarial Representation Learning and Graph Fusion Network for Multimodal Fusion Sijie Mai Haifeng Hu Songlong Xing GAN 69 185 0 18 Nov 2019
ALBERT: A Lite BERT for Self-supervised Learning of Language Representations Zhenzhong Lan Mingda Chen Sebastian Goodman Kevin Gimpel Piyush Sharma Radu Soricut SSL AIMat 274 6,420 0 26 Sep 2019
Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks Nils Reimers Iryna Gurevych 729 11,979 0 27 Aug 2019
RoBERTa: A Robustly Optimized BERT Pretraining Approach Yinhan Liu Myle Ott Naman Goyal Jingfei Du Mandar Joshi Danqi Chen Omer Levy M. Lewis Luke Zettlemoyer Veselin Stoyanov AIMat 421 24,160 0 26 Jul 2019
Multi-Task Semi-Supervised Adversarial Autoencoding for Speech Emotion Recognition S. Latif R. Rana Sara Khalifa Raja Jurdak J. Epps Björn W. Schuller 65 99 0 13 Jul 2019
XLNet: Generalized Autoregressive Pretraining for Language Understanding Zhilin Yang Zihang Dai Yiming Yang J. Carbonell Ruslan Salakhutdinov Quoc V. Le AI4CE 183 8,386 0 19 Jun 2019
Self-supervised Visual Feature Learning with Deep Neural Networks: A Survey Longlong Jing Yingli Tian SSL 100 1,692 0 16 Feb 2019
Found in Translation: Learning Robust Joint Representations by Cyclic Translations Between Modalities Hai Pham Paul Pu Liang Thomas Manzini Louis-Philippe Morency Barnabás Póczós 54 407 0 19 Dec 2018
DialogueRNN: An Attentive RNN for Emotion Detection in Conversations Navonil Majumder Soujanya Poria Devamanyu Hazarika Rada Mihalcea Alexander Gelbukh Min Zhang 48 717 0 01 Nov 2018
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding Jacob Devlin Ming-Wei Chang Kenton Lee Kristina Toutanova VLM SSL SSeg 1.1K 93,936 0 11 Oct 2018
MELD: A Multimodal Multi-Party Dataset for Emotion Recognition in Conversations Soujanya Poria Devamanyu Hazarika Navonil Majumder Gautam Naik Min Zhang Rada Mihalcea 85 1,055 0 05 Oct 2018
Investigation of Multimodal Features, Classifiers and Fusion Methods for Emotion Recognition Zheng Lian Ya Li J. Tao Jian Huang 20 22 0 13 Sep 2018