The Interspeech 2024 Challenge on Speech Processing Using Discrete Units

11 June 2024

Xuankai Chang

Jiatong Shi

Jinchuan Tian

Yuning Wu

Yuxun Tang

Yihan Wu

Shinji Watanabe

Papers citing "The Interspeech 2024 Challenge on Speech Processing Using Discrete Units"

42 / 42 papers shown

Title
APCodec: A Neural Audio Codec with Parallel Amplitude and Phase Spectrum Encoding and Decoding Yang Ai Xiao-Hang Jiang Ye-Xin Lu Hui-Peng Du Zhenhua Ling 41 25 0 16 Feb 2024
Seamless: Multilingual Expressive and Streaming Speech Translation Seamless Communication Loïc Barrault Yu-An Chung Mariano Coria Meglioli David Dale ... Paden Tomasello Changhan Wang Jeff Wang Skyler Wang Mary Williamson 56 155 0 08 Dec 2023
UniAudio: An Audio Foundation Model Toward Universal Audio Generation Dongchao Yang Jinchuan Tian Xuejiao Tan Rongjie Huang Songxiang Liu ... Jiang Bian Xixin Wu Zhou Zhao Shinji Watanabe Helen M. Meng CVBM AuLLM 74 127 0 01 Oct 2023
Exploring Speech Recognition, Translation, and Understanding with Discrete Speech Units: A Comparative Study Xuankai Chang Brian Yan Kwanghee Choi Jee-weon Jung Yichen Lu ... Pengcheng Guo Yao-Fei Cheng Pavel Denisov Kohei Saijo Hsiu-Hsuan Wang 101 40 0 27 Sep 2023
Voxtlm: unified decoder-only models for consolidating speech recognition/synthesis and speech/text continuation tasks Soumi Maiti Yifan Peng Shukjae Choi Jee-weon Jung Xuankai Chang Shinji Watanabe VLM AuLLM 76 65 0 14 Sep 2023
EXPRESSO: A Benchmark and Analysis of Discrete Expressive Speech Resynthesis Tu Nguyen Wei-Ning Hsu Antony DÁvirro Bowen Shi Itai Gat ... Gabriel Synnaeve Michael Hassid Felix Kreuk Yossi Adi Emmanuel Dupoux 59 60 0 10 Aug 2023
Many-to-Many Spoken Language Translation via Unified Speech and Text Representation Learning with Unit-to-Unit Translation Minsu Kim J. Choi Dahun Kim Y. Ro 67 10 0 03 Aug 2023
AudioPaLM: A Large Language Model That Can Speak and Listen Paul Kishan Rubenstein Chulayuth Asawaroengchai D. Nguyen Ankur Bapna Zalan Borsos ... Neil Zeghidour Yu Zhang Zhishuai Zhang Lukás Zilka Christian Frank LM&MA AuLLM VLM 96 286 0 22 Jun 2023
High-Fidelity Audio Compression with Improved RVQGAN Rithesh Kumar Prem Seetharaman Alejandro Luebs I. Kumar Kundan Kumar 88 326 0 11 Jun 2023
Simple and Controllable Music Generation Jade Copet Felix Kreuk Itai Gat Tal Remez David Kant Gabriel Synnaeve Yossi Adi Alexandre Défossez MGen 105 367 0 08 Jun 2023
Exploration of Efficient End-to-End ASR using Discretized Input from Self-Supervised Learning Xuankai Chang Brian Yan Yuya Fujita Takashi Maekaku Shinji Watanabe 56 38 0 29 May 2023
Textually Pretrained Speech Language Models Michael Hassid Tal Remez Tu Nguyen Itai Gat Alexis Conneau ... Alexandre Défossez Gabriel Synnaeve Emmanuel Dupoux Roy Schwartz Yossi Adi VLM SyDa 83 58 0 22 May 2023
ML-SUPERB: Multilingual Speech Universal PERformance Benchmark Jiatong Shi Dan Berrebbi William Chen Ho-Lam Chung En-Pei Hu ... Xuankai Chang Shang-Wen Li Abdel-rahman Mohamed Hung-yi Lee Shinji Watanabe ELM 92 66 0 18 May 2023
Enhancing Speech-to-Speech Translation with Multiple TTS Targets Jiatong Shi Yun Tang Ann Lee Hirofumi Inaguma Changhan Wang J. Pino Shinji Watanabe 61 9 0 10 Apr 2023
ESPnet-ST-v2: Multipurpose Spoken Language Translation Toolkit Brian Yan Jiatong Shi Yun Tang Hirofumi Inaguma Yifan Peng ... Zhaoheng Ni Moto Hira Soumi Maiti J. Pino Shinji Watanabe 48 20 0 10 Apr 2023
Neural Codec Language Models are Zero-Shot Text to Speech Synthesizers Chengyi Wang Sanyuan Chen Yu-Huan Wu Zi-Hua Zhang Long Zhou ... Huaming Wang Jinyu Li Lei He Sheng Zhao Furu Wei 168 706 0 05 Jan 2023
Robust Speech Recognition via Large-Scale Weak Supervision Alec Radford Jong Wook Kim Tao Xu Greg Brockman C. McLeavey Ilya Sutskever OffRL 178 3,655 0 06 Dec 2022
E-Branchformer: Branchformer with Enhanced merging for speech recognition Kwangyoun Kim Felix Wu Yifan Peng Jing Pan Prashant Sridhar Kyu Jeong Han Shinji Watanabe 104 115 0 30 Sep 2022
AudioGen: Textually Guided Audio Generation Felix Kreuk Gabriel Synnaeve Adam Polyak Uriel Singer Alexandre Défossez Jade Copet Devi Parikh Yaniv Taigman Yossi Adi DiffM 61 308 0 30 Sep 2022
Muskits: an End-to-End Music Processing Toolkit for Singing Voice Synthesis Jiatong Shi Shuai Guo Tao Qian Nan Huo Tomoki Hayashi ... Xuankai Chang Hua-Wei Li Peter Wu Shinji Watanabe Qin Jin VLM 85 27 0 09 May 2022
UTMOS: UTokyo-SaruLab System for VoiceMOS Challenge 2022 Takaaki Saeki Detai Xin Wataru Nakata Tomoki Koriyama Shinnosuke Takamichi Hiroshi Saruwatari 90 207 0 05 Apr 2022
Generative Spoken Dialogue Language Modeling Tu Nguyen Eugene Kharitonov Jade Copet Yossi Adi Wei-Ning Hsu ... Paden Tomasello Robin Algayres Benoît Sagot Abdel-rahman Mohamed Emmanuel Dupoux AuLLM 87 88 0 30 Mar 2022
The VoiceMOS Challenge 2022 Wen-Chin Huang Erica Cooper Yu Tsao Hsin-Min Wang Tomoki Toda Junichi Yamagishi 37 106 0 21 Mar 2022
Opencpop: A High-Quality Open Source Chinese Popular Song Corpus for Singing Voice Synthesis Yu Wang Xinsheng Wang Pengcheng Zhu Jie Wu Hanzhao Li Heyang Xue Yongmao Zhang Lei Xie Mengxiao Bi 62 102 0 19 Jan 2022
Textless Speech Emotion Conversion using Discrete and Decomposed Representations Felix Kreuk Adam Polyak Jade Copet Eugene Kharitonov Tu Nguyen M. Rivière Wei-Ning Hsu Abdel-rahman Mohamed Emmanuel Dupoux Yossi Adi 63 33 0 14 Nov 2021
WavLM: Large-Scale Self-Supervised Pre-Training for Full Stack Speech Processing Sanyuan Chen Chengyi Wang Zhengyang Chen Yu-Huan Wu Shujie Liu ... Yao Qian Jian Wu Micheal Zeng Xiangzhan Yu Furu Wei SSL 239 1,857 0 26 Oct 2021
VISinger: Variational Inference with Adversarial Learning for End-to-End Singing Voice Synthesis Yongmao Zhang Jian Cong Heyang Xue Lei Xie Pengcheng Zhu Mengxiao Bi 56 75 0 17 Oct 2021
Text-Free Prosody-Aware Generative Spoken Language Modeling Eugene Kharitonov Ann Lee Adam Polyak Yossi Adi Jade Copet ... Tu Nguyen M. Rivière Abdel-rahman Mohamed Emmanuel Dupoux Wei-Ning Hsu 58 121 0 07 Sep 2021
Direct speech-to-speech translation with discrete units Ann Lee Peng-Jen Chen Changhan Wang Jiatao Gu Sravya Popuri ... Yossi Adi Qing He Yun Tang J. Pino Wei-Ning Hsu 62 188 0 12 Jul 2021
HuBERT: Self-Supervised Speech Representation Learning by Masked Prediction of Hidden Units Wei-Ning Hsu Benjamin Bolte Yao-Hung Hubert Tsai Kushal Lakhotia Ruslan Salakhutdinov Abdel-rahman Mohamed SSL 163 2,949 0 14 Jun 2021
Speech Resynthesis from Discrete Disentangled Self-Supervised Representations Adam Polyak Yossi Adi Jade Copet Eugene Kharitonov Kushal Lakhotia Wei-Ning Hsu Abdel-rahman Mohamed Emmanuel Dupoux 80 318 0 01 Apr 2021
Generative Spoken Language Modeling from Raw Audio Kushal Lakhotia Evgeny Kharitonov Wei-Ning Hsu Yossi Adi Adam Polyak ... Tu Nguyen Jade Copet Alexei Baevski A. Mohamed Emmanuel Dupoux AuLLM 246 358 0 01 Feb 2021
Recent Developments on ESPnet Toolkit Boosted by Conformer Pengcheng Guo Florian Boyer Xuankai Chang Tomoki Hayashi Yosuke Higuchi ... Jing Shi Shinji Watanabe Kun Wei Wangyou Zhang Yuekai Zhang 63 263 0 26 Oct 2020
wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representations Alexei Baevski Henry Zhou Abdel-rahman Mohamed Michael Auli SSL 280 5,790 0 20 Jun 2020
XiaoiceSing: A High-Quality and Integrated Singing Voice Synthesis System Peiling Lu Jie Wu Jian Luan Xu Tan Li Zhou 57 98 0 11 Jun 2020
FastSpeech 2: Fast and High-Quality End-to-End Text to Speech Yi Ren Chenxu Hu Xu Tan Tao Qin Sheng Zhao Zhou Zhao Tie-Yan Liu 105 1,396 0 08 Jun 2020
Conformer: Convolution-augmented Transformer for Speech Recognition Anmol Gulati James Qin Chung-Cheng Chiu Niki Parmar Yu Zhang ... Wei Han Shibo Wang Zhengdong Zhang Yonghui Wu Ruoming Pang 220 3,131 0 16 May 2020
DiscreTalk: Text-to-Speech as a Machine Translation Problem Tomoki Hayashi Shinji Watanabe 54 32 0 12 May 2020
ESPnet-TTS: Unified, Reproducible, and Integratable Open Source End-to-End Text-to-Speech Toolkit Tomoki Hayashi Ryuichi Yamamoto Katsuki Inoue Takenori Yoshimura Shinji Watanabe Tomoki Toda K. Takeda Yu Zhang Xu Tan VLM 85 205 0 24 Oct 2019
ESPnet: End-to-End Speech Processing Toolkit Shinji Watanabe Takaaki Hori Shigeki Karita Tomoki Hayashi Jiro Nishitoba ... Jahn Heymann Sanjeev Khudanpur Nanxin Chen Adithya Renduchintala Tsubasa Ochiai VLM 102 1,504 0 30 Mar 2018
Attention-Based Models for Speech Recognition J. Chorowski Dzmitry Bahdanau Dmitriy Serdyuk Kyunghyun Cho Yoshua Bengio 121 2,607 0 24 Jun 2015
Sequence Transduction with Recurrent Neural Networks Alex Graves 187 1,868 0 14 Nov 2012