v1v2 (latest)

Recent Trends of Multimodal Affective Computing: A Survey from NLP Perspective

11 September 2024

Erik Cambria

Hasti Seifi

ArXiv (abs)PDF HTML

Papers citing "Recent Trends of Multimodal Affective Computing: A Survey from NLP Perspective"

50 / 74 papers shown

Title
Tracing Intricate Cues in Dialogue: Joint Graph Structure and Sentiment Dynamics for Multimodal Emotion Recognition Jiang Li Francisco Alcántara-Ávila Zhigang Zeng 25 0 0 31 Jul 2024
Norface: Improving Facial Expression Analysis by Identity Normalization Hanwei Liu Rudong An Zhimeng Zhang Bowen Ma Wei Zhang Yan Song Yujing Hu Wei Chen Yu-qiong Ding CVBM 21 4 0 22 Jul 2024
FineCLIPER: Multi-modal Fine-grained CLIP for Dynamic Facial Expression Recognition with AdaptERs Haodong Chen Haojian Huang Junhao Dong Mingzhe Zheng Dian Shao 77 16 0 02 Jul 2024
CARAT: Contrastive Feature Reconstruction and Aggregation for Multi-Modal Multi-Label Emotion Recognition Cheng Peng Ke Chen Lidan Shou Gang Chen 66 8 0 15 Dec 2023
A Transformer-Based Model With Self-Distillation for Multimodal Emotion Recognition in Conversations Hui Ma Jian Wang Hongfei Lin Bo Zhang Yijia Zhang Bo Xu 80 46 0 31 Oct 2023
M2DF: Multi-grained Multi-curriculum Denoising Framework for Multimodal Aspect-based Sentiment Analysis Fei Zhao Chunhui Li Zhen Wu Yawen Ouyang Jianbing Zhang Xinyu Dai 89 18 0 23 Oct 2023
Leveraging Label Information for Multimodal Emotion Recognition Pei-Hsin Wang Sunlu Zeng Junqing Chen Lu Fan Meng Chen Youzheng Wu Xiaodong He 76 5 0 05 Sep 2023
InstructBLIP: Towards General-purpose Vision-Language Models with Instruction Tuning Wenliang Dai Junnan Li Dongxu Li A. M. H. Tiong Junqi Zhao Weisheng Wang Boyang Albert Li Pascale Fung Steven C. H. Hoi MLLM VLM 145 2,098 0 11 May 2023
LOGO-Former: Local-Global Spatio-Temporal Transformer for Dynamic Facial Expression Recognition Fuyan Ma Bin Sun Shutao Li ViT 60 21 0 05 May 2023
BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Models Junnan Li Dongxu Li Silvio Savarese Steven C. H. Hoi VLM MLLM 432 4,656 0 30 Jan 2023
Multimodal Deep Learning Cem Akkus Jiquan Ngiam Vladana Djakovic Steffen Jauch-Walser A. Khosla ... Jann Goschenhofer Honglak Lee A. Ng Daniel Schalk Matthias Aßenmacher 123 3,176 0 12 Jan 2023
Emotion Recognition with Pre-Trained Transformers Using Multimodal Signals Juan Vazquez-Rodriguez G. Lefebvre Julien Cumin James L. Crowley 96 12 0 22 Dec 2022
UniMSE: Towards Unified Multimodal Sentiment Analysis and Emotion Recognition Guimin Hu Ting-En Lin Yi Zhao Guangming Lu Yuchuan Wu Yongbin Li 108 121 0 21 Nov 2022
Scaling Instruction-Finetuned Language Models Hyung Won Chung Le Hou Shayne Longpre Barret Zoph Yi Tay ... Jacob Devlin Adam Roberts Denny Zhou Quoc V. Le Jason W. Wei ReLM LRM 231 3,158 0 20 Oct 2022
Video-based Cross-modal Auxiliary Network for Multimodal Sentiment Analysis Rongfei Chen Wenju Zhou Yang Li Huiyu Zhou 59 19 0 30 Aug 2022
Image as a Foreign Language: BEiT Pretraining for All Vision and Vision-Language Tasks Wenhui Wang Hangbo Bao Li Dong Johan Bjorck Zhiliang Peng ... Kriti Aggarwal O. Mohammed Saksham Singhal Subhojit Som Furu Wei MLLM VLM ViT 148 644 0 22 Aug 2022
Make Acoustic and Visual Cues Matter: CH-SIMS v2.0 Dataset and AV-Mixup Consistent Module Yih-Ling Liu Ziqi Yuan Huisheng Mao Zhiyun Liang Wanqiuyue Yang Yuanzhe Qiu Tie Cheng Xiaoteng Li Hua Xu Kai Gao 80 46 0 22 Aug 2022
Multimodal Speech Emotion Recognition using Cross Attention with Aligned Audio and Text Yoonhyung Lee Seunghyun Yoon Kyomin Jung 135 21 0 26 Jul 2022
M3ED: Multi-modal Multi-scene Multi-label Emotional Dialogue Database Jinming Zhao Tenggan Zhang Jingwen Hu Yuchen Liu Qin Jin Xinchao Wang Haizhou Li 71 56 0 09 May 2022
CoCa: Contrastive Captioners are Image-Text Foundation Models Jiahui Yu Zirui Wang Vijay Vasudevan Legg Yeung Mojtaba Seyedhosseini Yonghui Wu VLM CLIP OffRL 177 1,309 0 04 May 2022
Flamingo: a Visual Language Model for Few-Shot Learning Jean-Baptiste Alayrac Jeff Donahue Pauline Luc Antoine Miech Iain Barr ... Mikolaj Binkowski Ricardo Barreira Oriol Vinyals Andrew Zisserman Karen Simonyan MLLM VLM 418 3,610 0 29 Apr 2022
Trusted Multi-View Classification with Dynamic Evidential Fusion Zongbo Han Changqing Zhang Huazhu Fu Qiufeng Wang EDL 73 230 0 25 Apr 2022
Vision-Language Pre-Training for Multimodal Aspect-Based Sentiment Analysis Yan Ling Jianfei Yu Rui Xia 51 75 0 17 Apr 2022
UniDU: Towards A Unified Generative Dialogue Understanding Framework Zhi Chen Lu Chen B. Chen Libo Qin Yuncong Liu Su Zhu Jian-Guang Lou Kai Yu 72 13 0 10 Apr 2022
C3KG: A Chinese Commonsense Conversation Knowledge Graph Dawei Li Yanran Li Jiayi Zhang K. Li Chen Wei Jianwei Cui Bin Wang 74 14 0 06 Apr 2022
Incorporating Dynamic Semantics into Pre-Trained Language Model for Aspect-based Sentiment Analysis Kai Zhang Kunpeng Zhang Mengdi Zhang Hongke Zhao Qi Liu Wei Wu Enhong Chen 36 52 0 30 Mar 2022
MM-DFN: Multimodal Dynamic Fusion Network for Emotion Recognition in Conversations Dou Hu Xiaolong Hou Lingwei Wei Lian-Xin Jiang Yang Mo 87 125 0 04 Mar 2022
BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generation Junnan Li Dongxu Li Caiming Xiong Guosheng Lin MLLM BDL VLM CLIP 557 4,421 0 28 Jan 2022
UnifiedSKG: Unifying and Multi-Tasking Structured Knowledge Grounding with Text-to-Text Language Models Tianbao Xie Chen Henry Wu Peng Shi Ruiqi Zhong Torsten Scholak ... Lingpeng Kong Rui Zhang Noah A. Smith Luke Zettlemoyer Tao Yu LMTD 108 304 0 16 Jan 2022
Multimodal Representations Learning Based on Mutual Information Maximization and Minimization and Identity Embedding for Multimodal Sentiment Analysis Jiahao Zheng Sen Zhang Xiaoping Wang Zhigang Zeng 26 7 0 10 Jan 2022
Which is Making the Contribution: Modulating Unimodal and Cross-modal Dynamics for Multimodal Sentiment Analysis Ying Zeng Sijie Mai Haifeng Hu 70 19 0 10 Nov 2021
VLMo: Unified Vision-Language Pre-Training with Mixture-of-Modality-Experts Hangbo Bao Wenhui Wang Li Dong Qiang Liu Owais Khan Mohammed Kriti Aggarwal Subhojit Som Furu Wei VLM MLLM MoE 102 559 0 03 Nov 2021
UniMS: A Unified Framework for Multimodal Summarization with Knowledge Distillation Zhengkun Zhang Xiaojun Meng Yasheng Wang Xin Jiang Qun Liu Zhenglu Yang 77 47 0 13 Sep 2021
Finetuned Language Models Are Zero-Shot Learners Jason W. Wei Maarten Bosma Vincent Zhao Kelvin Guu Adams Wei Yu Brian Lester Nan Du Andrew M. Dai Quoc V. Le ALM UQCV 251 3,789 0 03 Sep 2021
Improving Multimodal Fusion with Hierarchical Mutual Information Maximization for Multimodal Sentiment Analysis Wei Han Hui Chen Soujanya Poria 74 338 0 01 Sep 2021
Align before Fuse: Vision and Language Representation Learning with Momentum Distillation Junnan Li Ramprasaath R. Selvaraju Akhilesh Deepak Gotmare Shafiq Joty Caiming Xiong Guosheng Lin FaML 223 1,979 0 16 Jul 2021
MMGCN: Multimodal Fusion via Deep Graph Convolution Network for Emotion Recognition in Conversation Jingwen Hu Yuchen Liu Jinming Zhao Qin Jin 77 209 0 14 Jul 2021
Transfer-based adaptive tree for multimodal sentiment analysis based on user latent aspects Sana Rahmani Saeid Hosseini R. Zall M. Kangavari Sara Kamran Wenlan Hua 52 22 0 27 Jun 2021
A Unified Generative Framework for Aspect-Based Sentiment Analysis Hang Yan Junqi Dai Tuo Ji Xipeng Qiu Zheng Zhang 73 283 0 08 Jun 2021
VATT: Transformers for Multimodal Self-Supervised Learning from Raw Video, Audio and Text Hassan Akbari Liangzhe Yuan Rui Qian Wei-Hong Chuang Shih-Fu Chang Huayu Chen Boqing Gong ViT 333 590 0 22 Apr 2021
CTNet: Context-based Tandem Network for Semantic Segmentation Zechao Li Yanpeng Sun Jinhui Tang 60 176 0 20 Apr 2021
The MuSe 2021 Multimodal Sentiment Analysis Challenge: Sentiment, Emotion, Physiological-Emotion, and Stress Lukas Stappen Alice Baird Lukas Christ Lea Schumann Benjamin Sertolli Eva-Maria Messner Min Zhang Guoying Zhao Björn W. Schuller 46 88 0 14 Apr 2021
AST: Audio Spectrogram Transformer Yuan Gong Yu-An Chung James R. Glass ViT 145 884 0 05 Apr 2021
Learning Transferable Visual Models From Natural Language Supervision Alec Radford Jong Wook Kim Chris Hallacy Aditya A. Ramesh Gabriel Goh ... Amanda Askell Pamela Mishkin Jack Clark Gretchen Krueger Ilya Sutskever CLIP VLM 1.0K 29,926 0 26 Feb 2021
Learning Modality-Specific Representations with Self-Supervised Multi-Task Learning for Multimodal Sentiment Analysis Wenmeng Yu Hua Xu Ziqi Yuan Jiele Wu SSL 110 465 0 09 Feb 2021
Prefix-Tuning: Optimizing Continuous Prompts for Generation Xiang Lisa Li Percy Liang 252 4,305 0 01 Jan 2021
An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale Alexey Dosovitskiy Lucas Beyer Alexander Kolesnikov Dirk Weissenborn Xiaohua Zhai ... Matthias Minderer G. Heigold Sylvain Gelly Jakob Uszkoreit N. Houlsby ViT 684 41,563 0 22 Oct 2020
Training Strategies to Handle Missing Modalities for Audio-Visual Expression Recognition Srinivas Parthasarathy Shiva Sundaram 76 78 0 02 Oct 2020
Language Models are Few-Shot Learners Tom B. Brown Benjamin Mann Nick Ryder Melanie Subbiah Jared Kaplan ... Christopher Berner Sam McCandlish Alec Radford Ilya Sutskever Dario Amodei BDL 904 42,463 0 28 May 2020
MISA: Modality-Invariant and -Specific Representations for Multimodal Sentiment Analysis Devamanyu Hazarika Roger Zimmermann Soujanya Poria 88 711 0 07 May 2020