Listening and Seeing Again: Generative Error Correction for Audio-Visual Speech Recognition

3 January 2025

Papers citing "Listening and Seeing Again: Generative Error Correction for Audio-Visual Speech Recognition"

32 / 32 papers shown

Title
MMGER: Multi-modal and Multi-granularity Generative Error Correction with LLM for Joint Accent and Speech Recognition Bingshen Mu Yangze Li Qijie Shao Kun Wei Xucheng Wan Naijun Zheng Huan Zhou Lei Xie 145 9 0 06 May 2024
Look, Listen, and Answer: Overcoming Biases for Audio-Visual Question Answering Jie Ma Min Hu Pinghui Wang Wangchun Sun Lingyun Song Hongbin Pei Jun Liu Youtian Du 161 7 0 18 Apr 2024
It's Never Too Late: Fusing Acoustic Information into Large Language Models for Automatic Speech Recognition Chen Chen Ruizhe Li Yuchen Hu Sabato Marco Siniscalchi Pin-Yu Chen Ensiong Chng Chao-Han Huck Yang 99 22 0 08 Feb 2024
Large Language Models are Efficient Learners of Noise-Robust Speech Recognition Yuchen Hu Chen Chen Chao-Han Huck Yang Ruizhe Li Chao Zhang Pin-Yu Chen Ensiong Chng 102 25 0 19 Jan 2024
SECap: Speech Emotion Captioning with Large Language Model Yaoxun Xu Hangting Chen Jianwei Yu Qiaochu Huang Zhiyong Wu Shixiong Zhang Guangzhi Li Yi Luo Rongzhi Gu 112 27 0 16 Dec 2023
SALMONN: Towards Generic Hearing Abilities for Large Language Models Changli Tang Wenyi Yu Guangzhi Sun Xianzhao Chen Tian Tan Wei Li Lu Lu Zejun Ma Chao Zhang LM&MA AuLLM 115 264 0 20 Oct 2023
Whispering LLaMA: A Cross-Modal Generative Error Correction Framework for Speech Recognition S. Radhakrishnan Chao-Han Huck Yang S. Khan Rohit Kumar N. Kiani D. Gómez-Cabrero Jesper N. Tegnér 138 53 0 10 Oct 2023
HyPoradise: An Open Baseline for Generative Speech Recognition with Large Language Models Cheng Chen Yuchen Hu Chao-Han Huck Yang Sabato Marco Siniscalchi Pin-Yu Chen Eng Siong Chng 99 48 0 27 Sep 2023
Connecting Speech Encoder and Large Language Model for ASR Wenyi Yu Changli Tang Guangzhi Sun Xianzhao Chen T. Tan Wei Li Lu Lu Zejun Ma Chao Zhang AuLLM 80 77 0 25 Sep 2023
Robust Visual Question Answering: Datasets, Methods, and Future Challenges Jie Ma Pinghui Wang Dechen Kong Zewei Wang Jun Liu Hongbin Pei Junzhou Zhao OOD 126 23 0 21 Jul 2023
Llama 2: Open Foundation and Fine-Tuned Chat Models Hugo Touvron Louis Martin Kevin R. Stone Peter Albert Amjad Almahairi ... Sharan Narang Aurelien Rodriguez Robert Stojnic Sergey Edunov Thomas Scialom AI4MH ALM 562 12,138 0 18 Jul 2023
Video-ChatGPT: Towards Detailed Video Understanding via Large Vision and Language Models Muhammad Maaz H. Rasheed Salman Khan Fahad Shahbaz Khan MLLM 164 662 0 08 Jun 2023
Video-LLaMA: An Instruction-tuned Audio-Visual Language Model for Video Understanding Hang Zhang Xin Li Lidong Bing MLLM 226 1,068 0 05 Jun 2023
ImageBind: One Embedding Space To Bind Them All Rohit Girdhar Alaaeldin El-Nouby Zhuang Liu Mannat Singh Kalyan Vasudev Alwala Armand Joulin Ishan Misra VLM 219 945 0 09 May 2023
Auto-AVSR: Audio-Visual Speech Recognition with Automatic Labels Pingchuan Ma A. Haliassos Adriana Fernandez-Lopez Honglie Chen Stavros Petridis Maja Pantic 96 115 0 25 Mar 2023
LLaMA: Open and Efficient Foundation Language Models Hugo Touvron Thibaut Lavril Gautier Izacard Xavier Martinet Marie-Anne Lachaux ... Faisal Azhar Aurelien Rodriguez Armand Joulin Edouard Grave Guillaume Lample ALM PILM 1.7K 13,558 0 27 Feb 2023
BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Models Junnan Li Dongxu Li Silvio Savarese Steven C. H. Hoi VLM MLLM 634 4,679 0 30 Jan 2023
Leveraging Modality-specific Representations for Audio-visual Speech Recognition via Reinforcement Learning Chen Chen Yuchen Hu Qiang Zhang Heqing Zou Beier Zhu Eng Siong Chng 90 28 0 10 Dec 2022
Robust Speech Recognition via Large-Scale Weak Supervision Alec Radford Jong Wook Kim Tao Xu Greg Brockman C. McLeavey Ilya Sutskever OffRL 233 3,780 0 06 Dec 2022
Flamingo: a Visual Language Model for Few-Shot Learning Jean-Baptiste Alayrac Jeff Donahue Pauline Luc Antoine Miech Iain Barr ... Mikolaj Binkowski Ricardo Barreira Oriol Vinyals Andrew Zisserman Karen Simonyan MLLM VLM 431 3,622 0 29 Apr 2022
VideoMAE: Masked Autoencoders are Data-Efficient Learners for Self-Supervised Video Pre-Training Zhan Tong Yibing Song Jue Wang Limin Wang ViT 252 1,222 0 23 Mar 2022
Robust Self-Supervised Audio-Visual Speech Recognition Bowen Shi Wei-Ning Hsu Abdel-rahman Mohamed 104 93 0 05 Jan 2022
Learning Audio-Visual Speech Representation by Masked Multimodal Cluster Prediction Bowen Shi Wei-Ning Hsu Kushal Lakhotia Abdel-rahman Mohamed SSL 141 321 0 05 Jan 2022
FastCorrect 2: Fast Error Correction on Multiple Candidates for Automatic Speech Recognition Yichong Leng Xu Tan Rui Wang Linchen Zhu Jin Xu ... Linquan Liu Tao Qin Xiang-Yang Li Ed Lin Tie-Yan Liu 129 42 0 29 Sep 2021
LoRA: Low-Rank Adaptation of Large Language Models J. E. Hu Yelong Shen Phillip Wallis Zeyuan Allen-Zhu Yuanzhi Li Shean Wang Lu Wang Weizhu Chen OffRL AI4TS AI4CE ALM AIMat 828 10,659 0 17 Jun 2021
HuBERT: Self-Supervised Speech Representation Learning by Masked Prediction of Hidden Units Wei-Ning Hsu Benjamin Bolte Yao-Hung Hubert Tsai Kushal Lakhotia Ruslan Salakhutdinov Abdel-rahman Mohamed SSL 193 3,017 0 14 Jun 2021
End-to-end Audio-visual Speech Recognition with Conformers Pingchuan Ma Stavros Petridis Maja Pantic 160 234 0 12 Feb 2021
Conformer: Convolution-augmented Transformer for Speech Recognition Anmol Gulati James Qin Chung-Cheng Chiu Niki Parmar Yu Zhang ... Wei Han Shibo Wang Zhengdong Zhang Yonghui Wu Ruoming Pang 236 3,179 0 16 May 2020
Attention Is All You Need Ashish Vaswani Noam M. Shazeer Niki Parmar Jakob Uszkoreit Llion Jones Aidan Gomez Lukasz Kaiser Illia Polosukhin 3DV 1.0K 133,589 0 12 Jun 2017
Central Moment Discrepancy (CMD) for Domain-Invariant Representation Learning Werner Zellinger Thomas Grubinger E. Lughofer T. Natschläger Susanne Saminger-Platz OOD 110 582 0 28 Feb 2017
MUSAN: A Music, Speech, and Noise Corpus David Snyder Guoguo Chen Daniel Povey 92 1,358 0 28 Oct 2015
Neural Machine Translation of Rare Words with Subword Units Rico Sennrich Barry Haddow Alexandra Birch 291 7,779 0 31 Aug 2015