Cross-modal Contrastive Learning for Speech Translation

5 May 2022

Lei Li

Papers citing "Cross-modal Contrastive Learning for Speech Translation"

23 / 23 papers shown

Title
Speech Translation Refinement using Large Language Models Huaixia Dou Xinyu Tian Xinglin Lyu Jie Zhu Junhui Li Lifan Guo 143 0 0 28 Jan 2025
WhiSPA: Semantically and Psychologically Aligned Whisper with Self-Supervised Contrastive and Student-Teacher Learning Rajath Rao Adithya V Ganesan O. Kjell Jonah Luby Akshay Raghavan ... B. Luft Camilo Ruggero Neville Ryant R. Kotov H. A. Schwartz 32 0 0 15 Jan 2025
CoSTA: Code-Switched Speech Translation using Aligned Speech-Text Interleaving Bhavani Shankar P. Jyothi Pushpak Bhattacharyya 45 1 0 16 Jun 2024
Enhancing Multilingual Voice Toxicity Detection with Speech-Text Alignment Joseph Liu Mahesh Kumar Nandwana Janne Pylkkönen Hannes Heikinheimo Morgan McGuire 37 1 0 14 Jun 2024
CWCL: Cross-Modal Transfer with Continuously Weighted Contrastive Loss R. S. Srinivasa Jaejin Cho Chouchang Yang Yashas Malur Saidutta Ching Hua Lee Yilin Shen Hongxia Jin VLM 34 8 0 26 Sep 2023
End-to-End Simultaneous Speech Translation with Differentiable Segmentation Shaolei Zhang Yang Feng 20 17 0 25 May 2023
Improving speech translation by fusing speech and text Wenbiao Yin Zhicheng Liu Chengqi Zhao Tao Wang Jian-Fei Tong Rong Ye 15 4 0 23 May 2023
DUB: Discrete Unit Back-translation for Speech Translation Dong Zhang Rong Ye Tom Ko Mingxuan Wang Yaqian Zhou 13 23 0 19 May 2023
Back Translation for Speech-to-text Translation Without Transcripts Qingkai Fang Yang Feng 30 13 0 15 May 2023
Improving Speech Translation by Cross-Modal Multi-Grained Contrastive Learning Hao Zhang Nianwen Si Yaqi Chen Wenlin Zhang Xukui Yang Dan Qu Weiqiang Zhang 35 9 0 20 Apr 2023
SegAugment: Maximizing the Utility of Speech Translation Data with Segmentation-based Augmentations Ioannis Tsiamas José A. R. Fonollosa Marta R. Costa-jussá 41 6 0 19 Dec 2022
WACO: Word-Aligned Contrastive Learning for Speech Translation Siqi Ouyang Rong Ye Lei Li 29 25 0 19 Dec 2022
AdaTranS: Adapting with Boundary-based Shrinking for End-to-End Speech Translation Xingshan Zeng Liangyou Li Qun Liu 24 5 0 17 Dec 2022
Improving End-to-end Speech Translation by Leveraging Auxiliary Speech and Text Data Yuhao Zhang Chen Xu Bojie Hu Chunliang Zhang Tong Xiao Jingbo Zhu 18 15 0 04 Dec 2022
Bootstrapping meaning through listening: Unsupervised learning of spoken sentence embeddings Jian Zhu Zuoyu Tian Yadong Liu Cong Zhang Chia-wen Lo SSL 30 2 0 23 Oct 2022
Low-resource Neural Machine Translation with Cross-modal Alignment Zhe Yang Qingkai Fang Yang Feng VLM 37 9 0 13 Oct 2022
CoBERT: Self-Supervised Speech Representation Learning Through Code Representation Learning Chutong Meng Junyi Ao Tom Ko Mingxuan Wang Haizhou Li SSL 41 6 0 08 Oct 2022
Learning Decoupled Retrieval Representation for Nearest Neighbour Neural Machine Translation Qiang Wang Rongxiang Weng Ming-Tso Chen RALM 27 4 0 19 Sep 2022
Speechformer: Reducing Information Loss in Direct Speech Translation Sara Papi Marco Gaido Matteo Negri Marco Turchi 56 23 0 09 Sep 2021
ESimCSE: Enhanced Sample Building Method for Contrastive Learning of Unsupervised Sentence Embedding Xing Wu Chaochen Gao Liangjun Zang Jizhong Han Zhongyuan Wang Songlin Hu SSL AILaw 33 129 0 09 Sep 2021
Improving Speech Translation by Understanding and Learning from the Auxiliary Text Translation Task Yun Tang J. Pino Xian Li Changhan Wang Dmitriy Genzel 106 81 0 12 Jul 2021
Unified Vision-Language Pre-Training for Image Captioning and VQA Luowei Zhou Hamid Palangi Lei Zhang Houdong Hu Jason J. Corso Jianfeng Gao MLLM VLM 252 927 0 24 Sep 2019
End-to-End Automatic Speech Translation of Audiobooks Alexandre Berard Laurent Besacier A. Kocabiyikoglu Olivier Pietquin 75 190 0 12 Feb 2018