Large Language Models are Strong Audio-Visual Speech Recognition Learners

Large Language Models are Strong Audio-Visual Speech Recognition Learners

18 September 2024

Umberto Cappellazzo

Honglie Chen

Stavros Petridis

Daniele Falavigna

Alessio Brutti

Maja Pantic

Papers citing "Large Language Models are Strong Audio-Visual Speech Recognition Learners"

5 / 5 papers shown

Title
MMS-LLaMA: Efficient LLM-based Audio-Visual Speech Recognition with Minimal Multimodal Speech Tokens Jeong Hun Yeo Hyeongseop Rha Se Jin Park Y. Ro 51 0 0 14 Mar 2025
Adaptive Audio-Visual Speech Recognition via Matryoshka-Based Multimodal LLMs Umberto Cappellazzo Minsu Kim Stavros Petridis 54 0 0 09 Mar 2025
Audio-Visual Representation Learning via Knowledge Distillation from Speech Foundation Models Jing-Xuan Zhang Genshun Wan Jianqing Gao Zhen-Hua Ling 47 0 0 09 Feb 2025
mWhisper-Flamingo for Multilingual Audio-Visual Noise-Robust Speech Recognition Andrew Rouditchenko Saurabhchand Bhati Samuel Thomas Hilde Kuehne Rogerio Feris 111 1 0 03 Feb 2025
Multi-modal Speech Transformer Decoders: When Do Multiple Modalities Improve Accuracy? Yiwen Guan V. Trinh Vivek Voleti Jacob Whitehill 34 1 0 13 Sep 2024