UnitY: Two-pass Direct Speech-to-speech Translation with Discrete Units

15 December 2022

Papers citing "UnitY: Two-pass Direct Speech-to-speech Translation with Discrete Units"

20 / 20 papers shown

Title
Universal Speech Token Learning via Low-Bitrate Neural Codec and Pretrained Representations Xue Jiang Xiulian Peng Yuan Zhang Yan-Heng Lu SSL 83 0 0 15 Mar 2025
High-Fidelity Simultaneous Speech-To-Speech Translation Tom Labiausse Laurent Mazaré Edouard Grave P. Pérez Alexandre Défossez Neil Zeghidour 171 0 0 05 Feb 2025
Discrete Speech Unit Extraction via Independent Component Analysis Tomohiko Nakamura Kwanghee Choi Keigo Hojo Yoshiaki Bando Satoru Fukayama Shinji Watanabe 43 0 0 11 Jan 2025
CTC-based Non-autoregressive Textless Speech-to-Speech Translation Qingkai Fang Zhengrui Ma Yan Zhou Min Zhang Yang Feng 52 0 0 11 Jun 2024
Can We Achieve High-quality Direct Speech-to-Speech Translation without Parallel Speech Data? Qingkai Fang Shaolei Zhang Zhengrui Ma Min Zhang Yang Feng VLM 43 1 0 11 Jun 2024
A Non-autoregressive Generation Framework for End-to-End Simultaneous Speech-to-Any Translation Zhengrui Ma Qingkai Fang Shaolei Zhang Shoutao Guo Yang Feng Min Zhang 53 9 0 11 Jun 2024
Textless Acoustic Model with Self-Supervised Distillation for Noise-Robust Expressive Speech-to-Speech Translation Min-Jae Hwang Ilia Kulikov Benjamin Peloquin Hongyu Gong Peng-Jen Chen Ann Lee 29 1 0 04 Jun 2024
Direct Punjabi to English speech translation using discrete units Prabhjot Kaur L. A. M. Bush Weisong Shi 31 0 0 25 Feb 2024
GSQA: An End-to-End Model for Generative Spoken Question Answering Min-Han Shih Ho-Lam Chung Yu-Chi Pai Ming-Hao Hsu Guan-Ting Lin Shang-Wen Li Hung-yi Lee ELM AuLLM 33 2 0 15 Dec 2023
Learning When to Speak: Latency and Quality Trade-offs for Simultaneous Speech-to-Speech Translation with Offline Models Liam Dugan Anshul Wadhawan Kyle Spence Chris Callison-Burch Morgan McGuire Victor Zordan OffRL 30 2 0 01 Jun 2023
DUB: Discrete Unit Back-translation for Speech Translation Dong Zhang Rong Ye Tom Ko Mingxuan Wang Yaqian Zhou 21 23 0 19 May 2023
Back Translation for Speech-to-text Translation Without Transcripts Qingkai Fang Yang Feng 35 13 0 15 May 2023
ESPnet-ST-v2: Multipurpose Spoken Language Translation Toolkit Brian Yan Jiatong Shi Yun Tang H. Inaguma Yifan Peng ... Zhaoheng Ni Moto Hira Soumi Maiti J. Pino Shinji Watanabe 19 20 0 10 Apr 2023
Leveraging Pseudo-labeled Data to Improve Direct Speech-to-Speech Translation Qianqian Dong Fengpeng Yue Tom Ko Mingxuan Wang Qibing Bai Yu Zhang 34 16 0 18 May 2022
Consistent Human Evaluation of Machine Translation across Language Pairs Daniel Licht Cynthia Gao Janice Lam Francisco Guzman Mona T. Diab Philipp Koehn 32 17 0 17 May 2022
Fast-MD: Fast Multi-Decoder End-to-End Speech Translation with Non-Autoregressive Hidden Intermediates H. Inaguma Siddharth Dalmia Brian Yan Shinji Watanabe 65 11 0 27 Sep 2021
Improving Speech Translation by Understanding and Learning from the Auxiliary Text Translation Task Yun Tang J. Pino Xian Li Changhan Wang Dmitriy Genzel 106 81 0 12 Jul 2021
Searchable Hidden Intermediates for End-to-End Models of Decomposable Sequence Tasks Siddharth Dalmia Brian Yan Vikas Raunak Florian Metze Shinji Watanabe 37 30 0 02 May 2021
Tied Multitask Learning for Neural Speech Translation Antonios Anastasopoulos David Chiang 100 171 0 19 Feb 2018
End-to-End Automatic Speech Translation of Audiobooks Alexandre Berard Laurent Besacier A. Kocabiyikoglu Olivier Pietquin 75 190 0 12 Feb 2018