Fused Acoustic and Text Encoding for Multimodal Bilingual Pretraining and Speech Translation

10 February 2021

Papers citing "Fused Acoustic and Text Encoding for Multimodal Bilingual Pretraining and Speech Translation"

28 / 28 papers shown

Title
RCRank: Multimodal Ranking of Root Causes of Slow Queries in Cloud Database Systems Biao Ouyang Yingying Zhang Hanyin Cheng Yang Shu Chenjuan Guo Bin Yang Qingsong Wen L. Fan Christian S. Jensen 53 1 0 06 Mar 2025
EMMeTT: Efficient Multimodal Machine Translation Training Piotr Żelasko Zhehuai Chen Mengru Wang Daniel Galvez Oleksii Hrinchuk Shuoyang Ding Ke Hu Jagadeesh Balam Vitaly Lavrukhin Boris Ginsburg 40 1 0 20 Sep 2024
Recent Advances in Direct Speech-to-text Translation Chen Xu Rong Ye Qianqian Dong Chengqi Zhao Tom Ko Mingxuan Wang Tong Xiao Jingbo Zhu 21 18 0 20 Jun 2023
ChatBridge: Bridging Modalities with Large Language Model as a Language Catalyst Zijia Zhao Longteng Guo Tongtian Yue Si-Qing Chen Shuai Shao Xinxin Zhu Zehuan Yuan Jing Liu MLLM 40 52 0 25 May 2023
ComSL: A Composite Speech-Language Model for End-to-End Speech-to-Text Translation Chenyang Le Yao Qian Long Zhou Shujie Liu Yanmin Qian Michael Zeng Xuedong Huang 24 13 0 24 May 2023
Improving speech translation by fusing speech and text Wenbiao Yin Zhicheng Liu Chengqi Zhao Tao Wang Jian-Fei Tong Rong Ye 15 4 0 23 May 2023
DUB: Discrete Unit Back-translation for Speech Translation Dong Zhang Rong Ye Tom Ko Mingxuan Wang Yaqian Zhou 21 23 0 19 May 2023
SLTUNET: A Simple Unified Model for Sign Language Translation Biao Zhang Mathias Müller Rico Sennrich SLR 43 33 0 02 May 2023
Efficient CTC Regularization via Coarse Labels for End-to-End Speech Translation Biao Zhang Barry Haddow Rico Sennrich 17 3 0 21 Feb 2023
Transformadores: Fundamentos teoricos y Aplicaciones J. D. L. Torre 78 0 0 18 Feb 2023
Hierarchical Cross-modal Transformer for RGB-D Salient Object Detection Hao Chen Feihong Shen ViT 36 0 0 16 Feb 2023
WACO: Word-Aligned Contrastive Learning for Speech Translation Siqi Ouyang Rong Ye Lei Li 32 25 0 19 Dec 2022
Improving End-to-end Speech Translation by Leveraging Auxiliary Speech and Text Data Yuhao Zhang Chen Xu Bojie Hu Chunliang Zhang Tong Xiao Jingbo Zhu 27 15 0 04 Dec 2022
ERNIE-SAT: Speech and Text Joint Pretraining for Cross-Lingual Multi-Speaker Text-to-Speech Xiaoran Fan Chao Pang Tian Yuan Richard He Bai Renjie Zheng ... Junkun Chen Zeyu Chen Liang Huang Yu Sun Hua Wu 40 0 0 07 Nov 2022
SpeechUT: Bridging Speech and Text with Hidden-Unit for Encoder-Decoder Based Speech-Text Pre-training Zi-Hua Zhang Long Zhou Junyi Ao Shujie Liu Lirong Dai Jinyu Li Furu Wei 61 57 0 07 Oct 2022
M-Adapter: Modality Adaptation for End-to-End Speech-to-Text Translation Jinming Zhao Haomiao Yang Ehsan Shareghi Gholamreza Haffari 48 19 0 03 Jul 2022
Multimodal Learning with Transformers: A Survey P. Xu Xiatian Zhu David A. Clifton ViT 72 528 0 13 Jun 2022
Building Machine Translation Systems for the Next Thousand Languages Ankur Bapna Isaac Caswell Julia Kreutzer Orhan Firat D. Esch ... Apurva Shah Yanping Huang Z. Chen Yonghui Wu Macduff Hughes 56 98 0 09 May 2022
An Analysis of Semantically-Aligned Speech-Text Embeddings M. Huzaifah Ivan Kukanov 30 7 0 04 Apr 2022
Leveraging unsupervised and weakly-supervised data to improve direct speech-to-speech translation Ye Jia Yifan Ding Ankur Bapna Colin Cherry Yu Zhang Alexis Conneau Nobuyuki Morioka 47 20 0 24 Mar 2022
STEMM: Self-learning with Speech-text Manifold Mixup for Speech Translation Qingkai Fang Rong Ye Lei Li Yang Feng Mingxuan Wang 35 95 0 20 Mar 2022
mSLAM: Massively multilingual joint pre-training for speech and text Ankur Bapna Colin Cherry Yu Zhang Ye Jia Melvin Johnson Yong Cheng Simran Khanuja Jason Riesa Alexis Conneau VLM 30 111 0 03 Feb 2022
SLAM: A Unified Encoder for Speech and Language Modeling via Speech-Text Joint Pre-Training Ankur Bapna Yu-An Chung Na Wu Anmol Gulati Ye Jia J. Clark Melvin Johnson Jason Riesa Alexis Conneau Yu Zhang VLM 61 94 0 20 Oct 2021
Direct Simultaneous Speech-to-Text Translation Assisted by Synchronized Streaming ASR Junkun Chen Mingbo Ma Renjie Zheng Liang Huang 31 31 0 11 Jun 2021
Searchable Hidden Intermediates for End-to-End Models of Decomposable Sequence Tasks Siddharth Dalmia Brian Yan Vikas Raunak Florian Metze Shinji Watanabe 39 30 0 02 May 2021
End-to-end Speech Translation via Cross-modal Progressive Training Rong Ye Mingxuan Wang Lei Li 28 71 0 21 Apr 2021
GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding Alex Jinpeng Wang Amanpreet Singh Julian Michael Felix Hill Omer Levy Samuel R. Bowman ELM 297 6,984 0 20 Apr 2018
End-to-End Automatic Speech Translation of Audiobooks Alexandre Berard Laurent Besacier A. Kocabiyikoglu Olivier Pietquin 75 190 0 12 Feb 2018