data2vec: A General Framework for Self-supervised Learning in Speech, Vision and Language

7 February 2022

Papers citing "data2vec: A General Framework for Self-supervised Learning in Speech, Vision and Language"

50 / 174 papers shown

Title
TS-SUPERB: A Target Speech Processing Benchmark for Speech Self-Supervised Learning Models Junyi Peng Takanori Ashihara Marc Delcroix Tsubasa Ochiai Oldrich Plchot Shoko Araki J. Černocký ELM 24 0 0 10 May 2025
seq-JEPA: Autoregressive Predictive Learning of Invariant-Equivariant World Models Hafez Ghaemi Eilif Muller Shahab Bakhtiari 49 0 0 06 May 2025
Contextures: Representations from Contexts Runtian Zhai Kai Yang Che-Ping Tsai Burak Varici Zico Kolter Pradeep Ravikumar 101 0 0 02 May 2025
A Survey on Music Generation from Single-Modal, Cross-Modal, and Multi-Modal Perspectives Shuyu Li Shulei Ji Zihao W. Wang Songruoyao Wu Jiaxing Yu K. Zhang MGen VGen 70 1 0 01 Apr 2025
Heterogeneous bimodal attention fusion for speech emotion recognition Jiachen Luo Huy Phan Lin Wang Joshua Reiss 44 0 0 09 Mar 2025
The order in speech disorder: a scoping review of state of the art machine learning methods for clinical speech classification Birger Moëll Fredrik Sand Aronsson Per Östberg Jonas Beskow 34 0 0 03 Mar 2025
MIM-Refiner: A Contrastive Learning Boost from Intermediate Pre-Trained Representations Benedikt Alkin Lukas Miklautz Sepp Hochreiter Johannes Brandstetter VLM 63 8 0 24 Feb 2025
Graph Perceiver IO: A General Architecture for Graph Structured Data Seyun Bae Hoyoon Byun Changdae Oh Yoon-Sik Cho Kyungwoo Song GNN 90 2 0 24 Feb 2025
ConceptVAE: Self-Supervised Fine-Grained Concept Disentanglement from 2D Echocardiographies C. Ciușdel Alex Serban Tiziano Passerini CoGe 69 1 0 03 Feb 2025
Fine Tuning without Catastrophic Forgetting via Selective Low Rank Adaptation Reza Akbarian Bafghi Carden Bagwell Avinash Ravichandran Ashish Shrivastava M. Raissi 46 0 0 28 Jan 2025
Multi-Task Corrupted Prediction for Learning Robust Audio-Visual Speech Representation Sungnyun Kim Sungwoo Cho Sangmin Bae Kangwook Jang Se-Young Yun SSL 68 1 0 23 Jan 2025
Deep Neural Networks and Brain Alignment: Brain Encoding and Decoding (Survey) S. Oota Zijiao Chen Manish Gupta R. Bapi G. Jobard F. Alexandre X. Hinaut 3DV AI4CE 44 11 0 31 Dec 2024
Wearable Accelerometer Foundation Models for Health via Knowledge Distillation Salar Abbaspourazad Anshuman Mishra Joseph D. Futoma Andrew C. Miller Ian Shapiro 83 0 0 15 Dec 2024
Locality Alignment Improves Vision-Language Models Ian Covert Tony Sun James Y. Zou Tatsunori Hashimoto VLM 64 3 0 14 Oct 2024
Learn from Real: Reality Defender's Submission to ASVspoof5 Challenge Yi Zhu C. Goel Surya Koppisetti Trang Tran Ankur Kumar Gaurav Bharaj AAML 28 0 0 09 Oct 2024
Denoising with a Joint-Embedding Predictive Architecture Dengsheng Chen Jie Hu Xiaoming Wei Enhua Wu DiffM 50 2 0 02 Oct 2024
Self-supervised Speech Models for Word-Level Stuttered Speech Detection Yi-Jen Shih Zoi Gkalitsiou A. Dimakis David Harwath 37 1 0 16 Sep 2024
A Survey of the Self Supervised Learning Mechanisms for Vision Transformers Asifullah Khan A. Sohail M. Fiaz Mehdi Hassan Tariq Habib Afridi ... Muhammad Zaigham Zaheer Kamran Ali Tangina Sultana Ziaurrehman Tanoli Naeem Akhter 45 3 0 30 Aug 2024
WavTokenizer: an Efficient Acoustic Discrete Codec Tokenizer for Audio Language Modeling Shengpeng Ji Ziyue Jiang Xize Cheng Yifu Chen Minghui Fang ... Rongjie Huang Yidi Jiang Qian Chen Zhou Zhao Zhou Zhao VLM 52 33 0 29 Aug 2024
Overview of Speaker Modeling and Its Applications: From the Lens of Deep Speaker Representation Learning Shuai Wang Zheng-Shou Chen Kong Aik Lee Yan-min Qian Haizhou Li 30 4 0 21 Jul 2024
AnatoMask: Enhancing Medical Image Segmentation with Reconstruction-guided Self-masking Yuheng Li Tianyu Luan Yizhou Wu Shaoyan Pan Yenho Chen Xiaofeng Yang 35 4 0 09 Jul 2024
Emotion-Aware Speech Self-Supervised Representation Learning with Intensity Knowledge Rui Liu Zening Ma SSL 34 1 0 10 Jun 2024
On the social bias of speech self-supervised models Yi-Cheng Lin T. Lin Hsi-Che Lin Andy T. Liu Hung-yi Lee 37 3 0 07 Jun 2024
Investigating the Áutoencoder Behavior' in Speech Self-Supervised Models: a focus on HuBERT's Pretraining Valentin Vielzeuf SSL 36 0 0 14 May 2024
AdaKD: Dynamic Knowledge Distillation of ASR models using Adaptive Loss Weighting Shreyan Ganguly Roshan Nayak Rakshith Rao Ujan Deb AP Prathosh 24 1 0 11 May 2024
Fake it to make it: Using synthetic data to remedy the data shortage in joint multimodal speech-and-gesture synthesis Shivam Mehta Anna Deichler Jim O'Regan Birger Moëll Jonas Beskow G. Henter Simon Alexanderson 36 4 0 30 Apr 2024
Modeling Caption Diversity in Contrastive Vision-Language Pretraining Samuel Lavoie Polina Kirichenko Mark Ibrahim Mahmoud Assran Andrew Gordon Wilson Aaron Courville Nicolas Ballas CLIP VLM 59 19 0 30 Apr 2024
Point-JEPA: A Joint Embedding Predictive Architecture for Self-Supervised Learning on Point Cloud Ayumu Saito Prachi Kudeshia Jiju Poovvancheri 3DPC 33 7 0 25 Apr 2024
MAD Speech: Measures of Acoustic Diversity of Speech Matthieu Futeral A. Agostinelli Marco Tagliasacchi Neil Zeghidour Eugene Kharitonov 46 1 0 16 Apr 2024
A Large-Scale Evaluation of Speech Foundation Models Shu-Wen Yang Heng-Jui Chang Zili Huang Andy T. Liu Cheng-I Jeff Lai ... Kushal Lakhotia Shang-Wen Li Abdelrahman Mohamed Shinji Watanabe Hung-yi Lee 38 19 0 15 Apr 2024
OmniSat: Self-Supervised Modality Fusion for Earth Observation Guillaume Astruc Nicolas Gonthier Clement Mallet Loic Landrieu 30 25 0 12 Apr 2024
NeuroNet: A Novel Hybrid Self-Supervised Learning Framework for Sleep Stage Classification Using Single-Channel EEG Cheol-Hui Lee Hakseung Kim Hyun-jee Han Min-Kyung Jung Byung C. Yoon Dong-Joo Kim 32 5 0 10 Apr 2024
Siamese Vision Transformers are Scalable Audio-visual Learners Yan-Bo Lin Gedas Bertasius 37 5 0 28 Mar 2024
Unified Static and Dynamic Network: Efficient Temporal Filtering for Video Grounding Jingjing Hu Dan Guo Kun Li Zhan Si Xun Yang Xiaojun Chang Meng Wang 59 3 0 21 Mar 2024
A Comprehensive Review of Machine Learning Advances on Data Change: A Cross-Field Perspective Jeng-Lin Li Chih-Fan Hsu Ming-Ching Chang Wei-Chao Chen OOD 36 2 0 20 Feb 2024
TelME: Teacher-leading Multimodal Fusion Network for Emotion Recognition in Conversation Taeyang Yun Hyunkuk Lim Jeong-Hoon Lee Min Song 32 11 0 16 Jan 2024
MERBench: A Unified Evaluation Benchmark for Multimodal Emotion Recognition Zheng Lian Licai Sun Yong Ren Hao Gu Haiyang Sun Lan Chen Bin Liu Jianhua Tao 15 12 0 07 Jan 2024
CrisisViT: A Robust Vision Transformer for Crisis Image Classification Zijun Long R. McCreadie Muhammad Imran 72 9 0 05 Jan 2024
Morphing Tokens Draw Strong Masked Image Models Taekyung Kim Byeongho Heo Dongyoon Han 44 3 0 30 Dec 2023
emotion2vec: Self-Supervised Pre-Training for Speech Emotion Representation Ziyang Ma Zhisheng Zheng Jiaxin Ye Jinchao Li Zhifu Gao Shiliang Zhang Xie Chen MDE SLR SSL 25 85 0 23 Dec 2023
Efficiency-oriented approaches for self-supervised speech representation learning Luis Lugo Valentin Vielzeuf SSL 19 1 0 18 Dec 2023
4M: Massively Multimodal Masked Modeling David Mizrahi Roman Bachmann Ouguzhan Fatih Kar Teresa Yeo Mingfei Gao Afshin Dehghan Amir Zamir MLLM 39 62 0 11 Dec 2023
LiDAR: Sensing Linear Probing Performance in Joint Embedding SSL Architectures Vimal Thilak Chen Huang Omid Saremi Laurent Dinh Hanlin Goh Preetum Nakkiran Josh Susskind Etai Littwin 18 7 0 07 Dec 2023
Unsupervised Video Domain Adaptation with Masked Pre-Training and Collaborative Self-Training Arun V. Reddy William Paul Corban Rivera Ketul Shah Celso M. de Melo Rama Chellappa 37 4 0 05 Dec 2023
A-JEPA: Joint-Embedding Predictive Architecture Can Listen Zhengcong Fei Mingyuan Fan Junshi Huang 23 17 0 27 Nov 2023
SSIN: Self-Supervised Learning for Rainfall Spatial Interpolation Jia Li Yanyan Shen Lei Chen Charles Wang Wai Ng 17 3 0 27 Nov 2023
R-Spin: Efficient Speaker and Noise-invariant Representation Learning with Acoustic Pieces Heng-Jui Chang James R. Glass 33 3 0 15 Nov 2023
OmniVec: Learning robust representations with cross modal sharing Siddharth Srivastava Gaurav Sharma SSL 21 64 0 07 Nov 2023
Fast Word Error Rate Estimation Using Self-Supervised Representations for Speech and Text Chanho Park Chengsong Lu Mingjie Chen Thomas Hain 18 3 0 12 Oct 2023
Graph-level Representation Learning with Joint-Embedding Predictive Architectures Geri Skenderi Hang Li Jiliang Tang Marco Cristani AI4TS GNN 52 3 0 27 Sep 2023