AV-data2vec: Self-supervised Learning of Audio-Visual Speech
Representations with Contextualized Target Representations

AV-data2vec: Self-supervised Learning of Audio-Visual Speech Representations with Contextualized Target Representations

10 February 2023

Papers citing "AV-data2vec: Self-supervised Learning of Audio-Visual Speech Representations with Contextualized Target Representations"

16 / 16 papers shown

Title
CoGenAV: Versatile Audio-Visual Representation Learning via Contrastive-Generative Synchronization Detao Bai Zhiheng Ma Xihan Wei Liefeng Bo 141 0 0 06 May 2025
mWhisper-Flamingo for Multilingual Audio-Visual Noise-Robust Speech Recognition Andrew Rouditchenko Saurabhchand Bhati Samuel Thomas Hilde Kuehne Rogerio Feris 116 1 0 03 Feb 2025
Multi-Task Corrupted Prediction for Learning Robust Audio-Visual Speech Representation Sungnyun Kim Sungwoo Cho Sangmin Bae Kangwook Jang Se-Young Yun SSL 68 1 0 23 Jan 2025
Large Language Models are Strong Audio-Visual Speech Recognition Learners Umberto Cappellazzo Minsu Kim Honglie Chen Pingchuan Ma Stavros Petridis Daniele Falavigna Alessio Brutti Maja Pantic 36 9 0 18 Sep 2024
Stutter-Solver: End-to-end Multi-lingual Dysfluency Detection Xuanru Zhou Cheol Jun Cho Ayati Sharma Brittany Morin D. Baquirin ... Zachary Miller B. Tee M. G. Tempini Jiachen Lian Gopala Anumanchipalli 34 3 0 15 Sep 2024
Multilingual Audio-Visual Speech Recognition with Hybrid CTC/RNN-T Fast Conformer Maxime Burchi Krishna C. Puvvada Jagadeesh Balam Boris Ginsburg Radu Timofte 44 8 0 14 Mar 2024
Unconstrained Dysfluency Modeling for Dysfluent Speech Transcription and Detection Jiachen Lian Carly Feng Naasir Farooqi Steve Li Anshul Kashyap Cheol Jun Cho Peter Wu Robin Netzorg Tingle Li Gopala Krishna Anumanchipalli 40 13 0 20 Dec 2023
Articulatory Representation Learning Via Joint Factor Analysis and Neural Matrix Factorization Jiachen Lian A. Black Yijingxiu Lu L. Goldstein Shinji Watanabe Gopala K. Anumanchipalli 46 14 0 29 Oct 2022
SpeechUT: Bridging Speech and Text with Hidden-Unit for Encoder-Decoder Based Speech-Text Pre-training Zi-Hua Zhang Long Zhou Junyi Ao Shujie Liu Lirong Dai Jinyu Li Furu Wei 61 57 0 07 Oct 2022
Visual Speech Recognition for Multiple Languages in the Wild Pingchuan Ma Stavros Petridis M. Pantic VLM 128 144 0 26 Feb 2022
Transformer-Based Video Front-Ends for Audio-Visual Speech Recognition for Single and Multi-Person Video Dmitriy Serdyuk Otavio Braga Olivier Siohan ViT 91 40 0 25 Jan 2022
VideoCLIP: Contrastive Pre-training for Zero-shot Video-Text Understanding Hu Xu Gargi Ghosh Po-Yao (Bernie) Huang Dmytro Okhonko Armen Aghajanyan Florian Metze Luke Zettlemoyer Florian Metze Luke Zettlemoyer Christoph Feichtenhofer CLIP VLM 259 560 0 28 Sep 2021
Emerging Properties in Self-Supervised Vision Transformers Mathilde Caron Hugo Touvron Ishan Misra Hervé Jégou Julien Mairal Piotr Bojanowski Armand Joulin 350 5,811 0 29 Apr 2021
VATT: Transformers for Multimodal Self-Supervised Learning from Raw Video, Audio and Text Hassan Akbari Liangzhe Yuan Rui Qian Wei-Hong Chuang Shih-Fu Chang Huayu Chen Boqing Gong ViT 251 577 0 22 Apr 2021
CLIP4Clip: An Empirical Study of CLIP for End to End Video Clip Retrieval Huaishao Luo Lei Ji Ming Zhong Yang Chen Wen Lei Nan Duan Tianrui Li CLIP VLM 326 781 0 18 Apr 2021
VoxCeleb2: Deep Speaker Recognition Joon Son Chung Arsha Nagrani Andrew Zisserman 251 2,233 0 14 Jun 2018