Understanding and Bridging the Modality Gap for Speech Translation

15 May 2023

Papers citing "Understanding and Bridging the Modality Gap for Speech Translation"

17 / 17 papers shown

Title
Speech Translation Refinement using Large Language Models Huaixia Dou Xinyu Tian Xinglin Lyu Jie Zhu Junhui Li Lifan Guo 176 0 0 28 Jan 2025
CoSTA: Code-Switched Speech Translation using Aligned Speech-Text Interleaving Bhavani Shankar P. Jyothi Pushpak Bhattacharyya 48 1 0 16 Jun 2024
CTC-based Non-autoregressive Textless Speech-to-Speech Translation Qingkai Fang Zhengrui Ma Yan Zhou Min Zhang Yang Feng 52 0 0 11 Jun 2024
Can We Achieve High-quality Direct Speech-to-Speech Translation without Parallel Speech Data? Qingkai Fang Shaolei Zhang Zhengrui Ma Min Zhang Yang Feng VLM 43 1 0 11 Jun 2024
Pushing the Limits of Zero-shot End-to-End Speech Translation Ioannis Tsiamas Gerard I. Gállego José A. R. Fonollosa Marta R. Costa-jussá 43 7 0 16 Feb 2024
Rethinking and Improving Multi-task Learning for End-to-end Speech Translation Yuhao Zhang Chen Xu Bei Li Hao Chen Tong Xiao Chunliang Zhang Jingbo Zhu 26 5 0 07 Nov 2023
Unified Segment-to-Segment Framework for Simultaneous Sequence Generation Shaolei Zhang Yang Feng 23 7 0 27 Oct 2023
Bridging the Gap between Synthetic and Authentic Images for Multimodal Machine Translation Wenyu Guo Qingkai Fang Dong Yu Yang Feng 22 6 0 20 Oct 2023
DASpeech: Directed Acyclic Transformer for Fast and High-quality Speech-to-Speech Translation Qingkai Fang Yan Zhou Yangzhou Feng 40 6 0 11 Oct 2023
Tuning Large language model for End-to-end Speech Translation Hao Zhang Nianwen Si Yaqi Chen Wenlin Zhang Xu Yang Dan Qu Xiaolin Jiao 20 8 0 03 Oct 2023
Bridging the Gaps of Both Modality and Language: Synchronous Bilingual CTC for Speech Translation and Speech Recognition Chen Xu Xiaoqian Liu Erfeng He Yuhao Zhang Qianqian Dong Tong Xiao Jingbo Zhu Dapeng Man Wu Yang 32 0 0 21 Sep 2023
An Empirical Study of Consistency Regularization for End-to-End Speech-to-Text Translation Pengzhi Gao Ruiqing Zhang Zhongjun He Hua Wu Haifeng Wang 25 4 0 28 Aug 2023
Modality Adaption or Regularization? A Case Study on End-to-End Speech Translation Yucheng Han Chen Xu Tong Xiao Jingbo Zhu 27 3 0 13 Jun 2023
CMOT: Cross-modal Mixup via Optimal Transport for Speech Translation Yan Zhou Qingkai Fang Yang Feng OT 45 25 0 24 May 2023
Back Translation for Speech-to-text Translation Without Transcripts Qingkai Fang Yang Feng 35 13 0 15 May 2023
SpeechUT: Bridging Speech and Text with Hidden-Unit for Encoder-Decoder Based Speech-Text Pre-training Zi-Hua Zhang Long Zhou Junyi Ao Shujie Liu Lirong Dai Jinyu Li Furu Wei 61 57 0 07 Oct 2022
Improving Speech Translation by Understanding and Learning from the Auxiliary Text Translation Task Yun Tang J. Pino Xian Li Changhan Wang Dmitriy Genzel 108 81 0 12 Jul 2021