v1v2 (latest)

Sylber: Syllabic Embedding Representation of Speech from Raw Audio

9 October 2024

Cheol Jun Cho

Nicholas Lee

Akshat Gupta

Dhruv Agarwal

Ethan Chen

Alan W Black

Gopala K. Anumanchipalli

ArXiv (abs)PDF HTML

Papers citing "Sylber: Syllabic Embedding Representation of Speech from Raw Audio"

48 / 48 papers shown

Title
Self-Supervised Syllable Discovery Based on Speaker-Disentangled HuBERT Ryota Komatsu Takahiro Shinozaki SSL 86 1 0 16 Sep 2024
Articulatory Encodec: Coding Speech through Vocal Tract Kinematics Cheol Jun Cho Peter Wu Tejas S. Prabhune Dhruv Agarwal Gopala K. Anumanchipalli 84 7 0 18 Jun 2024
NAST: Noise Aware Speech Tokenization for Speech Language Models Shoval Messica Yossi Adi 63 7 0 16 Jun 2024
Self-Supervised Speech Representations are More Phonetic than Semantic Kwanghee Choi Ankita Pasad Tomohiko Nakamura Satoru Fukayama Karen Livescu Shinji Watanabe 81 19 0 12 Jun 2024
WavLLM: Towards Robust and Adaptive Speech Large Language Model Shujie Hu Long Zhou Shujie Liu Sanyuan Chen Hongkun Hao ... Xunying Liu Jinyu Li S. Sivasankaran Linquan Liu Furu Wei AuLLM 84 62 0 31 Mar 2024
Acoustic BPE for Speech Generation with Discrete Tokens Feiyu Shen Yiwei Guo Chenpeng Du Xie Chen Kai Yu 52 13 0 23 Oct 2023
Self-Supervised Models of Speech Infer Universal Articulatory Kinematics Cheol Jun Cho Abdelrahman Mohamed Alan W. Black Gopala K. Anumanchipalli SSL 52 11 0 16 Oct 2023
SD-HuBERT: Sentence-Level Self-Distillation Induces Syllabic Organization in HuBERT Cheol Jun Cho Abdelrahman Mohamed Shang-Wen Li Alan W. Black Gopala K. Anumanchipalli 87 9 0 16 Oct 2023
Generative Spoken Language Model based on continuous word-sized audio tokens Robin Algayres Yossi Adi Tu Nguyen Jade Copet Gabriel Synnaeve Benoît Sagot Emmanuel Dupoux AuLLM 92 16 0 08 Oct 2023
Exploring Speech Recognition, Translation, and Understanding with Discrete Speech Units: A Comparative Study Xuankai Chang Brian Yan Kwanghee Choi Jee-weon Jung Yichen Lu ... Pengcheng Guo Yao-Fei Cheng Pavel Denisov Kohei Saijo Hsiu-Hsuan Wang 107 40 0 27 Sep 2023
EXPRESSO: A Benchmark and Analysis of Discrete Expressive Speech Resynthesis Tu Nguyen Wei-Ning Hsu Antony DÁvirro Bowen Shi Itai Gat ... Gabriel Synnaeve Michael Hassid Felix Kreuk Yossi Adi Emmanuel Dupoux 71 60 0 10 Aug 2023
Voicebox: Text-Guided Multilingual Universal Speech Generation at Scale Matt Le Apoorv Vyas Bowen Shi Brian Karrer Leda Sari ... Mary Williamson Vimal Manohar Yossi Adi Jay Mahadeokar Wei-Ning Hsu AuLLM 97 300 0 23 Jun 2023
An Information-Theoretic Analysis of Self-supervised Discrete Representations of Speech Badr M. Abdullah Mohammed Maqsood Shaik Bernd Möbius Dietrich Klakow SSL 39 8 0 04 Jun 2023
LibriTTS-R: A Restored Multi-Speaker Text-to-Speech Corpus Yuma Koizumi Heiga Zen Shigeki Karita Yifan Ding Kohei Yatabe Nobuyuki Morioka M. Bacchiani Yu Zhang Wei Han Ankur Bapna 93 77 0 30 May 2023
Textually Pretrained Speech Language Models Michael Hassid Tal Remez Tu Nguyen Itai Gat Alexis Conneau ... Alexandre Défossez Gabriel Synnaeve Emmanuel Dupoux Roy Schwartz Yossi Adi VLM SyDa 87 60 0 22 May 2023
Syllable Discovery and Cross-Lingual Generalization in a Visually Grounded, Self-Supervised Speech Model Puyuan Peng Shang-Wen Li Okko Räsänen Abdel-rahman Mohamed David Harwath SSL VLM 67 7 0 19 May 2023
Neural Codec Language Models are Zero-Shot Text to Speech Synthesizers Chengyi Wang Sanyuan Chen Yu-Huan Wu Zi-Hua Zhang Long Zhou ... Huaming Wang Jinyu Li Lei He Sheng Zhao Furu Wei 178 710 0 05 Jan 2023
Analysing Discrete Self Supervised Speech Representation for Spoken Language Modeling Amitay Sicherman Yossi Adi 63 37 0 02 Jan 2023
Robust Speech Recognition via Large-Scale Weak Supervision Alec Radford Jong Wook Kim Tao Xu Greg Brockman C. McLeavey Ilya Sutskever OffRL 194 3,684 0 06 Dec 2022
Textless Direct Speech-to-Speech Translation with Discrete Speech Representation Xinjian Li Ye Jia Chung-Cheng Chiu 85 30 0 31 Oct 2022
Evaluating context-invariance in unsupervised speech representations Mark Hallap Emmanuel Dupoux Ewan Dunbar SSL 75 11 0 27 Oct 2022
Evidence of Vocal Tract Articulation in Self-Supervised Learning of Speech Cheol Jun Cho Peter Wu Abdel-rahman Mohamed Gopala K. Anumanchipalli 64 33 0 21 Oct 2022
Flow Matching for Generative Modeling Y. Lipman Ricky T. Q. Chen Heli Ben-Hamu Maximilian Nickel Matt Le OOD 207 1,307 0 06 Oct 2022
AudioLM: a Language Modeling Approach to Audio Generation Zalan Borsos Raphaël Marinier Damien Vincent Eugene Kharitonov Olivier Pietquin ... Dominik Roblek O. Teboul David Grangier Marco Tagliasacchi Neil Zeghidour AuLLM 154 608 0 07 Sep 2022
Self-Supervised Speech Representation Learning: A Review Abdel-rahman Mohamed Hung-yi Lee Lasse Borgholt Jakob Drachmann Havtorn Joakim Edin ... Shang-Wen Li Karen Livescu Lars Maaløe Tara N. Sainath Shinji Watanabe SSL AI4TS 208 365 0 21 May 2022
UTMOS: UTokyo-SaruLab System for VoiceMOS Challenge 2022 Takaaki Saeki Detai Xin Wataru Nakata Tomoki Koriyama Shinnosuke Takamichi Hiroshi Saruwatari 104 211 0 05 Apr 2022
WavLM: Large-Scale Self-Supervised Pre-Training for Full Stack Speech Processing Sanyuan Chen Chengyi Wang Zhengyang Chen Yu-Huan Wu Shujie Liu ... Yao Qian Jian Wu Micheal Zeng Xiangzhan Yu Furu Wei SSL 252 1,873 0 26 Oct 2021
Text-Free Prosody-Aware Generative Spoken Language Modeling Eugene Kharitonov Ann Lee Adam Polyak Yossi Adi Jade Copet ... Tu Nguyen M. Rivière Abdel-rahman Mohamed Emmanuel Dupoux Wei-Ning Hsu 68 121 0 07 Sep 2021
W2v-BERT: Combining Contrastive Learning and Masked Language Modeling for Self-Supervised Speech Pre-Training Yu-An Chung Yu Zhang Wei Han Chung-Cheng Chiu James Qin Ruoming Pang Yonghui Wu SSL VLM 56 427 0 07 Aug 2021
Direct speech-to-speech translation with discrete units Ann Lee Peng-Jen Chen Changhan Wang Jiatao Gu Sravya Popuri ... Yossi Adi Qing He Yun Tang J. Pino Wei-Ning Hsu 74 189 0 12 Jul 2021
HuBERT: Self-Supervised Speech Representation Learning by Masked Prediction of Hidden Units Wei-Ning Hsu Benjamin Bolte Yao-Hung Hubert Tsai Kushal Lakhotia Ruslan Salakhutdinov Abdel-rahman Mohamed SSL 180 2,966 0 14 Jun 2021
Unsupervised Speech Recognition Alexei Baevski Wei-Ning Hsu Alexis Conneau Michael Auli SSL 121 275 0 24 May 2021
SUPERB: Speech processing Universal PERformance Benchmark Shu-Wen Yang Po-Han Chi Yung-Sung Chuang Cheng-I Jeff Lai Kushal Lakhotia ... Shuyan Dong Shang-Wen Li Shinji Watanabe Abdel-rahman Mohamed Hung-yi Lee SSL 108 937 0 03 May 2021
Emerging Properties in Self-Supervised Vision Transformers Mathilde Caron Hugo Touvron Ishan Misra Hervé Jégou Julien Mairal Piotr Bojanowski Armand Joulin 694 6,079 0 29 Apr 2021
RoFormer: Enhanced Transformer with Rotary Position Embedding Jianlin Su Yu Lu Shengfeng Pan Ahmed Murtadha Bo Wen Yunfeng Liu 284 2,500 0 20 Apr 2021
Speech Resynthesis from Discrete Disentangled Self-Supervised Representations Adam Polyak Yossi Adi Jade Copet Eugene Kharitonov Kushal Lakhotia Wei-Ning Hsu Abdel-rahman Mohamed Emmanuel Dupoux 82 318 0 01 Apr 2021
Generative Spoken Language Modeling from Raw Audio Kushal Lakhotia Evgeny Kharitonov Wei-Ning Hsu Yossi Adi Adam Polyak ... Tu Nguyen Jade Copet Alexei Baevski A. Mohamed Emmanuel Dupoux AuLLM 251 360 0 01 Feb 2021
MLS: A Large-Scale Multilingual Dataset for Speech Research Vineel Pratap Qiantong Xu Anuroop Sriram Gabriel Synnaeve R. Collobert AuLLM 89 503 0 07 Dec 2020
The Zero Resource Speech Benchmark 2021: Metrics and baselines for unsupervised spoken language modeling Tu Nguyen Maureen de Seyssel Patricia Roze M. Rivière Evgeny Kharitonov Alexei Baevski Ewan Dunbar Emmanuel Dupoux SSL 124 107 0 23 Nov 2020
AISHELL-3: A Multi-speaker Mandarin TTS Corpus and the Baselines Yao Shi Hui Bu Xin Xu Shaojing Zhang Ming Li 83 222 0 22 Oct 2020
HiFi-GAN: Generative Adversarial Networks for Efficient and High Fidelity Speech Synthesis Jungil Kong Jaehyeon Kim Jaekyoung Bae 177 1,936 0 12 Oct 2020
wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representations Alexei Baevski Henry Zhou Abdel-rahman Mohamed Michael Auli SSL 285 5,801 0 20 Jun 2020
Bootstrap your own latent: A new approach to self-supervised Learning Jean-Bastien Grill Florian Strub Florent Altché Corentin Tallec Pierre Harvey Richemond ... M. G. Azar Bilal Piot Koray Kavukcuoglu Rémi Munos Michal Valko SSL 371 6,806 0 13 Jun 2020
Libri-Light: A Benchmark for ASR with Limited or No Supervision Jacob Kahn M. Rivière Weiyi Zheng Evgeny Kharitonov Qiantong Xu ... Tatiana Likhomanenko Gabriel Synnaeve Armand Joulin Abdel-rahman Mohamed Emmanuel Dupoux AuLLM 70 672 0 17 Dec 2019
Momentum Contrast for Unsupervised Visual Representation Learning Kaiming He Haoqi Fan Yuxin Wu Saining Xie Ross B. Girshick SSL 204 12,085 0 13 Nov 2019
LibriTTS: A Corpus Derived from LibriSpeech for Text-to-Speech Heiga Zen Viet Dang R. Clark Yu Zhang Ron J. Weiss Ye Jia Zhiwen Chen Yonghui Wu 104 954 0 05 Apr 2019
Attention Is All You Need Ashish Vaswani Noam M. Shazeer Niki Parmar Jakob Uszkoreit Llion Jones Aidan Gomez Lukasz Kaiser Illia Polosukhin 3DV 713 131,652 0 12 Jun 2017
Sequence Transduction with Recurrent Neural Networks Alex Graves 191 1,870 0 14 Nov 2012