SimulMT to SimulST: Adapting Simultaneous Text Translation to End-to-End Simultaneous Speech Translation

3 November 2020

Papers citing "SimulMT to SimulST: Adapting Simultaneous Text Translation to End-to-End Simultaneous Speech Translation"

50 / 62 papers shown

Title
SimulS2S-LLM: Unlocking Simultaneous Inference of Speech LLMs for Speech-to-Speech Translation Keqi Deng Wenxi Chen Xie Chen P. Woodland 43 0 0 22 Apr 2025
Joint Training And Decoding for Multilingual End-to-End Simultaneous Speech Translation Wuwei Huang Renren Jin Wen Zhang Jian Luan Bin Wang Deyi Xiong 69 1 0 14 Mar 2025
InfiniSST: Simultaneous Translation of Unbounded Speech with Large Language Model Siqi Ouyang Xi Xu Lei Li 56 1 0 04 Mar 2025
Large Language Models Are Read/Write Policy-Makers for Simultaneous Generation Shoutao Guo Shaolei Zhang Zhengrui Ma Yang Feng 31 0 0 03 Jan 2025
CA*: Addressing Evaluation Pitfalls in Computation-Aware Latency for Simultaneous Speech Translation Xi Xu Wenyuan Xu Siqi Ouyang Lei Li 38 0 0 21 Oct 2024
What does it take to get state of the art in simultaneous speech-to-speech translation? Vincent Wilmet Johnson Du 22 0 0 02 Sep 2024
FASST: Fast LLM-based Simultaneous Speech Translation Siqi Ouyang Xi Xu Chinmay Dandekar Lei Li 23 3 0 18 Aug 2024
Towards Achieving Human Parity on End-to-end Simultaneous Speech Translation via LLM Agent Shanbo Cheng Zhichao Huang Tom Ko Hang Li Ningxin Peng Lu Xu Qini Zhang 48 3 0 31 Jul 2024
NAIST Simultaneous Speech Translation System for IWSLT 2024 Yuka Ko Ryo Fukuda Yuta Nishikawa Yasumasa Kano Tomoya Yanagita ... Haotian Tan Makoto Sakai S. Sakti Katsuhito Sudoh Satoshi Nakamura 28 1 0 30 Jun 2024
BESTOW: Efficient and Streamable Speech Language Model with the Best of Two Worlds in GPT and T5 Zhehuai Chen He Huang Oleksii Hrinchuk Krishna C. Puvvada Nithin Rao Koluguri Piotr Żelasko Jagadeesh Balam Boris Ginsburg AuLLM RALM 38 10 0 28 Jun 2024
A Non-autoregressive Generation Framework for End-to-End Simultaneous Speech-to-Any Translation Zhengrui Ma Qingkai Fang Shaolei Zhang Shoutao Guo Yang Feng Min Zhang 53 9 0 11 Jun 2024
Agent-SiMT: Agent-assisted Simultaneous Machine Translation with Large Language Models Shoutao Guo Shaolei Zhang Zhengrui Ma Min Zhang Yang Feng LLMAG 33 1 0 11 Jun 2024
StreamAtt: Direct Streaming Speech-to-Text Translation with Attention-based Audio History Selection Sara Papi Marco Gaido Matteo Negri L. Bentivogli 64 4 0 10 Jun 2024
Label-Synchronous Neural Transducer for E2E Simultaneous Speech Translation Keqi Deng Philip C. Woodland 41 4 0 06 Jun 2024
Decoder-only Streaming Transformer for Simultaneous Translation Shoutao Guo Shaolei Zhang Yang Feng 33 4 0 06 Jun 2024
StreamSpeech: Simultaneous Speech-to-Speech Translation with Multi-task Learning Shaolei Zhang Qingkai Fang Shoutao Guo Zhengrui Ma Min Zhang Yang Feng 29 4 0 05 Jun 2024
LeaPformer: Enabling Linear Transformers for Autoregressive and Simultaneous Tasks via Learned Proportions Victor Agostinelli Sanghyun Hong Lizhong Chen KELM 43 1 0 18 May 2024
Simultaneous Masking, Not Prompting Optimization: A Paradigm Shift in Fine-tuning LLMs for Simultaneous Translation Matthew Raffel Victor Agostinelli Lizhong Chen 38 5 0 16 May 2024
Speech Translation with Speech Foundation Models and Large Language Models: What is There and What is Missing? Marco Gaido Sara Papi Matteo Negri L. Bentivogli 43 13 0 19 Feb 2024
Conversational SimulMT: Efficient Simultaneous Translation with Large Language Models Minghan Wang Thuy-Trang Vu Yuxia Wang Ehsan Shareghi Gholamreza Haffari 40 2 0 16 Feb 2024
Streaming Sequence Transduction through Dynamic Compression Weiting Tan Yunmo Chen Tongfei Chen Guanghui Qin Haoran Xu Heidi C. Zhang Benjamin Van Durme Philipp Koehn 24 2 0 02 Feb 2024
R-BI: Regularized Batched Inputs enhance Incremental Decoding Framework for Low-Latency Simultaneous Speech Translation Jiaxin Guo Zhanglin Wu Zongyao Li Hengchao Shang Daimeng Wei Xiaoyu Chen Zhiqiang Rao Shaojun Li Hao Yang 32 1 0 11 Jan 2024
Efficient Monotonic Multihead Attention Xutai Ma Anna Y. Sun Siqi Ouyang Hirofumi Inaguma Paden Tomasello 44 4 0 07 Dec 2023
Unified Segment-to-Segment Framework for Simultaneous Sequence Generation Shaolei Zhang Yang Feng 23 7 0 27 Oct 2023
Direct Models for Simultaneous Translation and Automatic Subtitling: FBK@IWSLT2023 Sara Papi Marco Gaido Matteo Negri 43 7 0 27 Sep 2023
Incremental Blockwise Beam Search for Simultaneous Speech Translation with Controllable Quality-Latency Tradeoff Peter Polák Brian Yan Shinji Watanabe A. Waibel Ondrej Bojar 28 9 0 20 Sep 2023
Token-Level Serialized Output Training for Joint Streaming ASR and ST Leveraging Textual Alignments Sara Papi Peidong Wan Junkun Chen Jian Xue Jinyu Li Yashesh Gaur 26 8 0 07 Jul 2023
Implicit Memory Transformer for Computationally Efficient Simultaneous Speech Translation Matthew Raffel Lizhong Chen 9 5 0 03 Jul 2023
Shiftable Context: Addressing Training-Inference Context Mismatch in Simultaneous Speech Translation Matthew Raffel Drew Penney Lizhong Chen 16 3 0 03 Jul 2023
Partitioning-Guided K-Means: Extreme Empty Cluster Resolution for Extreme Model Compression Tianhong Huang Victor Agostinelli Lizhong Chen MQ 15 0 0 24 Jun 2023
Tagged End-to-End Simultaneous Speech Translation Training using Simultaneous Interpretation Data Yuka Ko Ryo Fukuda Yuta Nishikawa Yasumasa Kano Katsuhito Sudoh Satoshi Nakamura 29 6 0 14 Jun 2023
Learning When to Speak: Latency and Quality Trade-offs for Simultaneous Speech-to-Speech Translation with Offline Models Liam Dugan Anshul Wadhawan Kyle Spence Chris Callison-Burch Morgan McGuire Victor Zordan OffRL 30 2 0 01 Jun 2023
End-to-End Simultaneous Speech Translation with Differentiable Segmentation Shaolei Zhang Yang Feng 23 17 0 25 May 2023
AlignAtt: Using Attention-based Audio-Translation Alignments as a Guide for Simultaneous Speech Translation Sara Papi Marco Turchi Matteo Negri 27 20 0 19 May 2023
Improving Autoregressive NLP Tasks via Modular Linearized Attention Victor Agostinelli Lizhong Chen 24 1 0 17 Apr 2023
ESPnet-ST-v2: Multipurpose Spoken Language Translation Toolkit Brian Yan Jiatong Shi Yun Tang Hirofumi Inaguma Yifan Peng ... Zhaoheng Ni Moto Hira Soumi Maiti J. Pino Shinji Watanabe 19 20 0 10 Apr 2023
LEAPT: Learning Adaptive Prefix-to-prefix Translation For Simultaneous Machine Translation Lei Lin Shuangtao Li Xiaodon Shi 21 5 0 21 Mar 2023
Adapting Offline Speech Translation Models for Streaming with Future-Aware Distillation and Inference Biao Fu Minpeng Liao Kai Fan Zhongqiang Huang Boxing Chen Yidong Chen Xiaodon Shi 48 8 0 14 Mar 2023
Attention as a Guide for Simultaneous Speech Translation Sara Papi Matteo Negri Marco Turchi 26 30 0 15 Dec 2022
Align, Write, Re-order: Explainable End-to-End Speech Translation via Operation Sequence Generation Motoi Omachi Brian Yan Siddharth Dalmia Yuya Fujita Shinji Watanabe LRM 25 3 0 11 Nov 2022
Streaming Parrotron for on-device speech-to-speech conversion Oleg Rybakov Fadi Biadsy Xia Zhang Liyang Jiang Phoenix Meadowlark Shivani Agrawal 24 3 0 25 Oct 2022
Information-Transport-based Policy for Simultaneous Translation Shaolei Zhang Yang Feng 51 47 0 22 Oct 2022
Joint Speech Translation and Named Entity Recognition Marco Gaido Sara Papi Matteo Negri Marco Turchi 33 3 0 21 Oct 2022
Over-Generation Cannot Be Rewarded: Length-Adaptive Average Lagging for Simultaneous Speech Translation Sara Papi Marco Gaido Matteo Negri Marco Turchi 44 46 0 12 Jun 2022
Efficient yet Competitive Speech Translation: FBK@IWSLT2022 Marco Gaido Sara Papi Dennis Fucci G. Fiameni Matteo Negri Marco Turchi 31 19 0 05 May 2022
Data-Driven Adaptive Simultaneous Machine Translation Guangxu Xun Mingbo Ma Yuchen Bian Xingyu Cai Jiaji Huang Renjie Zheng Junkun Chen Jiahong Yuan Kenneth Ward Church Liang Huang 17 0 0 27 Apr 2022
Blockwise Streaming Transformer for Spoken Language Understanding and Simultaneous Speech Translation Keqi Deng Shinji Watanabe Jiatong Shi Siddhant Arora 24 15 0 19 Apr 2022
Does Simultaneous Speech Translation need Simultaneous Models? Sara Papi Marco Gaido Matteo Negri Marco Turchi 41 26 0 08 Apr 2022
Exploring Continuous Integrate-and-Fire for Adaptive Simultaneous Speech Translation Chih-Chiang Chang Hung-yi Lee 27 13 0 22 Mar 2022
Anticipation-Free Training for Simultaneous Machine Translation Chih-Chiang Chang Shun-Po Chuang Hung-yi Lee 25 6 0 30 Jan 2022