Listen and Translate: A Proof of Concept for End-to-End Speech-to-Text Translation

6 December 2016

Olivier Pietquin

Papers citing "Listen and Translate: A Proof of Concept for End-to-End Speech-to-Text Translation"

50 / 179 papers shown

Title
IIITH-BUT system for IWSLT 2025 low-resource Bhojpuri to Hindi speech translation Bhavana Akkiraju Aishwarya Pothula Santosh Kesiraju Anil Kumar Vuppala 86 0 0 05 Jun 2025
HENT-SRT: Hierarchical Efficient Neural Transducer with Self-Distillation for Joint Speech Recognition and Translation A. Hussein Cihan Xiao Matthew Wiesner Dan Povey Leibny Paola García Sanjeev Khudanpur 12 0 0 02 Jun 2025
Speech-to-Text Translation with Phoneme-Augmented CoT: Enhancing Cross-Lingual Transfer in Low-Resource Scenarios Gerard I. Gállego Oriol Pareras Martí Cortada Garcia Lucas Takanori Javier Hernando LRM 17 0 0 30 May 2025
AdaST: Dynamically Adapting Encoder States in the Decoder for End-to-End Speech-to-Text Translation Wuwei Huang Dexin Wang Deyi Xiong 86 4 0 18 Mar 2025
Joint Training And Decoding for Multilingual End-to-End Simultaneous Speech Translation Wuwei Huang Renren Jin Wen Zhang Jian Luan Bin Wang Deyi Xiong 100 1 0 14 Mar 2025
Exploiting Vulnerabilities in Speech Translation Systems through Targeted Adversarial Attacks Chang-rui Liu Haolin Wu Xi Yang Kui Zhang Cong Wu Weinan Zhang Nenghai Yu Tianwei Zhang Qing Guo Jie Zhang AAML 62 0 0 02 Mar 2025
Aligner-Encoders: Self-Attention Transformers Can Be Self-Transducers Adam Stooke Rohit Prabhavalkar K. Sim P. M. Mengibar 185 0 0 06 Feb 2025
High-Fidelity Simultaneous Speech-To-Speech Translation Tom Labiausse Laurent Mazaré Edouard Grave P. Pérez Alexandre Défossez Neil Zeghidour 486 1 0 05 Feb 2025
When End-to-End is Overkill: Rethinking Cascaded Speech-to-Text Translation Anna Min Chenxu Hu Yi Ren Hang Zhao 107 1 0 01 Feb 2025
Prepending or Cross-Attention for Speech-to-Text? An Empirical Comparison Tsz Kin Lam Marco Gaido Sara Papi L. Bentivogli Barry Haddow 116 0 0 04 Jan 2025
LLM-Ref: Enhancing Reference Handling in Technical Writing with Large Language Models Kazi Ahmed Asif Fuad Lizhong Chen 119 0 0 01 Nov 2024
CTC-GMM: CTC guided modality matching for fast and accurate streaming speech translation Rui Zhao Jinyu Li Ruchao Fan Matt Post 82 2 0 07 Oct 2024
Making LLMs Better Many-to-Many Speech-to-Text Translators with Curriculum Learning Yexing Du Youcheng Pan Ziyang Ma Keqi Deng Yifan Yang Keqi Deng Xie Chen Yang Xiang Ming Liu Bing Qin LRM 151 9 0 29 Sep 2024
LLaST: Improved End-to-end Speech Translation System Leveraged by Large Language Models Xi Chen Songyang Zhang Qibing Bai Kai-xiang Chen Satoshi Nakamura AuLLM 99 7 0 22 Jul 2024
CoSTA: Code-Switched Speech Translation using Aligned Speech-Text Interleaving Bhavani Shankar Preethi Jyothi Pushpak Bhattacharyya 88 1 0 16 Jun 2024
Lightweight Audio Segmentation for Long-form Speech Translation Jaesong Lee Soyoon Kim Hanbyul Kim Joon Son Chung 59 0 0 15 Jun 2024
Simul-Whisper: Attention-Guided Streaming Whisper with Truncation Detection Haoyu Wang Guoqiang Hu Guodong Lin Wei-Qiang Zhang Jian Li 77 3 0 14 Jun 2024
Soft Language Identification for Language-Agnostic Many-to-One End-to-End Speech Translation Peidong Wang Jian Xue Jinyu Li Junkun Chen Aswin Shanmugam Subramanian 81 0 0 12 Jun 2024
StreamAtt: Direct Streaming Speech-to-Text Translation with Attention-based Audio History Selection Sara Papi Marco Gaido Matteo Negri L. Bentivogli 139 7 0 10 Jun 2024
SBAAM! Eliminating Transcript Dependency in Automatic Subtitling Marco Gaido Sara Papi Matteo Negri Mauro Cettolo L. Bentivogli 86 1 0 17 May 2024
Pushing the Limits of Zero-shot End-to-End Speech Translation Ioannis Tsiamas Gerard I. Gállego José A. R. Fonollosa Marta R. Costa-jussá 92 10 0 16 Feb 2024
Soft Alignment of Modality Space for End-to-end Speech Translation Yuhao Zhang Kaiqi Kou Bei Li Chen Xu Chunliang Zhang Tong Xiao Jingbo Zhu 78 2 0 18 Dec 2023
Efficient Monotonic Multihead Attention Xutai Ma Anna Y. Sun Siqi Ouyang Hirofumi Inaguma Paden Tomasello 65 4 0 07 Dec 2023
End-to-End Speech-to-Text Translation: A Survey Nivedita Sethiya Chandresh Kumar Maurya 104 8 0 02 Dec 2023
Rethinking and Improving Multi-task Learning for End-to-end Speech Translation Yuhao Zhang Chen Xu Bei Li Hao Chen Tong Xiao Chunliang Zhang Jingbo Zhu 72 7 0 07 Nov 2023
End-to-End Single-Channel Speaker-Turn Aware Conversational Speech Translation Juan Pablo Zuluaga Zhaocheng Huang Xing Niu Rohit Paturi S. Srinivasan Prashant Mathur Brian Thompson Marcello Federico BDL 74 2 0 01 Nov 2023
Towards a Deep Understanding of Multilingual End-to-End Speech Translation Haoran Sun Xiaohu Zhao Yikun Lei Shaolin Zhu Deyi Xiong 69 8 0 31 Oct 2023
Enhancing expressivity transfer in textless speech-to-speech translation J. Duret Benjamin O’Brien Yannick Esteve Titouan Parcollet 53 2 0 11 Oct 2023
Modular Speech-to-Text Translation for Zero-Shot Cross-Modal Transfer Paul-Ambroise Duquenne Holger Schwenk Benoît Sagot 104 3 0 05 Oct 2023
Enhancing End-to-End Conversational Speech Translation Through Target Language Context Utilization A. Hussein Brian Yan Antonios Anastasopoulos Shinji Watanabe Sanjeev Khudanpur 83 7 0 27 Sep 2023
Direct Models for Simultaneous Translation and Automatic Subtitling: FBK@IWSLT2023 Sara Papi Marco Gaido Matteo Negri 77 7 0 27 Sep 2023
Bridging the Gaps of Both Modality and Language: Synchronous Bilingual CTC for Speech Translation and Speech Recognition Chen Xu Xiaoqian Liu Erfeng He Yuhao Zhang Qianqian Dong Tong Xiao Jingbo Zhu Dapeng Man Wu Yang 74 0 0 21 Sep 2023
DiariST: Streaming Speech Translation with Speaker Diarization Muqiao Yang Naoyuki Kanda Xiaofei Wang Junkun Chen Peidong Wang Jian Xue Jinyu Li Takuya Yoshioka 63 7 0 14 Sep 2023
An Empirical Study of Consistency Regularization for End-to-End Speech-to-Text Translation Pengzhi Gao Ruiqing Zhang Zhongjun He Hua Wu Haifeng Wang 53 5 0 28 Aug 2023
Sparks of Large Audio Models: A Survey and Outlook S. Latif Moazzam Shoukat Fahad Shamshad Muhammad Usama Yi Ren ... Wenwu Wang Xulong Zhang Roberto Togneri Min Zhang Björn W. Schuller LM&MA AuLLM 186 39 0 24 Aug 2023
On decoder-only architecture for speech-to-text and large language model integration Jian Wu Yashesh Gaur Zhuo Chen Long Zhou Yilun Zhu ... Jinyu Li Shujie Liu Bo Ren Linquan Liu Yu-Huan Wu AuLLM 119 136 0 08 Jul 2023
Recent Advances in Direct Speech-to-text Translation Chen Xu Rong Ye Qianqian Dong Chengqi Zhao Tom Ko Mingxuan Wang Tong Xiao Jingbo Zhu 107 22 0 20 Jun 2023
Transfer Learning from Pre-trained Language Models Improves End-to-End Speech Summarization Kohei Matsuura Takanori Ashihara Takafumi Moriya Tomohiro Tanaka Takatomo Kano A. Ogawa Marc Delcroix 61 8 0 07 Jun 2023
CTC-based Non-autoregressive Speech Translation Chen Xu Xiaoqian Liu Xiaowen Liu Qingxuan Sun Yuhao Zhang ... Tom Ko Mingxuan Wang Tong Xiao Anxiang Ma Jingbo Zhu 62 11 0 27 May 2023
Bridging the Granularity Gap for Acoustic Modeling Chen Xu Yuhao Zhang Chengbo Jiao Xiaoqian Liu Chi Hu Xin Zeng Tong Xiao Anxiang Ma Huizhen Wang JingBo Zhu 56 6 0 27 May 2023
ComSL: A Composite Speech-Language Model for End-to-End Speech-to-Text Translation Chenyang Le Yao Qian Long Zhou Shujie Liu Yanmin Qian Michael Zeng Xuedong Huang 72 13 0 24 May 2023
CMOT: Cross-modal Mixup via Optimal Transport for Speech Translation Yan Zhou Qingkai Fang Yang Feng OT 111 30 0 24 May 2023
Improving speech translation by fusing speech and text Wenbiao Yin Zhicheng Liu Chengqi Zhao Tao Wang Jian-Fei Tong Rong Ye 52 4 0 23 May 2023
DUB: Discrete Unit Back-translation for Speech Translation Dong Zhang Rong Ye Tom Ko Mingxuan Wang Yaqian Zhou 90 27 0 19 May 2023
Towards Speech Dialogue Translation Mediating Speakers of Different Languages Shuichiro Shimizu Chenhui Chu Sheng Li Sadao Kurohashi Kyoto University 39 2 0 16 May 2023
Back Translation for Speech-to-text Translation Without Transcripts Qingkai Fang Yang Feng 65 14 0 15 May 2023
Understanding and Bridging the Modality Gap for Speech Translation Qingkai Fang Yang Feng 75 26 0 15 May 2023
Improving Cascaded Unsupervised Speech Translation with Denoising Back-translation Yu-Kuan Fu Liang-Hsuan Tseng Jiatong Shi Chen-An Li Tsung-Yuan Hsu Shinji Watanabe Hung-yi Lee 38 5 0 12 May 2023
Hybrid Transducer and Attention based Encoder-Decoder Modeling for Speech-to-Text Tasks Yun Tang Anna Y. Sun Hirofumi Inaguma Xinyue Chen Ning Dong Xutai Ma Paden Tomasello J. Pino 100 22 0 04 May 2023
Improving Speech Translation by Cross-Modal Multi-Grained Contrastive Learning Hao Zhang Nianwen Si Yaqi Chen Wenlin Zhang Xukui Yang Dan Qu Weiqiang Zhang 71 10 0 20 Apr 2023