CTC-based Compression for Direct Speech Translation

2 February 2021

Papers citing "CTC-based Compression for Direct Speech Translation"

42 / 42 papers shown

Title
On The Landscape of Spoken Language Models: A Comprehensive Survey Siddhant Arora Kai-Wei Chang Chung-Ming Chien Yifan Peng Haibin Wu Yossi Adi Emmanuel Dupoux Hung-yi Lee Karen Livescu Shinji Watanabe 65 5 0 11 Apr 2025
How do Multimodal Foundation Models Encode Text and Speech? An Analysis of Cross-Lingual and Cross-Modal Representations Hyunji Lee Danni Liu Supriti Sinhamahapatra Jan Niehues 126 0 0 21 Feb 2025
Prepending or Cross-Attention for Speech-to-Text? An Empirical Comparison Tsz Kin Lam Marco Gaido Sara Papi L. Bentivogli Barry Haddow 65 0 0 04 Jan 2025
CTC-GMM: CTC guided modality matching for fast and accurate streaming speech translation Rui Zhao Jinyu Li Ruchao Fan Matt Post 46 1 0 07 Oct 2024
How to Connect Speech Foundation Models and Large Language Models? What Matters and What Does Not Francesco Verdini Pierfrancesco Melucci Stefano Perna Francesco Cariaggi Marco Gaido ... Marek Kasztelnik L. Bentivogli Sébastien Bratières P. Merialdo Simone Scardapane AuLLM 35 0 0 25 Sep 2024
Can We Achieve High-quality Direct Speech-to-Speech Translation without Parallel Speech Data? Qingkai Fang Shaolei Zhang Zhengrui Ma Min Zhang Yang Feng VLM 54 1 0 11 Jun 2024
A Non-autoregressive Generation Framework for End-to-End Simultaneous Speech-to-Any Translation Zhengrui Ma Qingkai Fang Shaolei Zhang Shoutao Guo Yang Feng Min Zhang 64 10 0 11 Jun 2024
StreamAtt: Direct Streaming Speech-to-Text Translation with Attention-based Audio History Selection Sara Papi Marco Gaido Matteo Negri L. Bentivogli 99 4 0 10 Jun 2024
SBAAM! Eliminating Transcript Dependency in Automatic Subtitling Marco Gaido Sara Papi Matteo Negri Mauro Cettolo L. Bentivogli 60 1 0 17 May 2024
Compact Speech Translation Models via Discrete Speech Units Pretraining Tsz Kin Lam Alexandra Birch Barry Haddow 83 2 0 29 Feb 2024
How do Hyenas deal with Human Speech? Speech Recognition and Translation with ConfHyena Marco Gaido Sara Papi Matteo Negri L. Bentivogli 66 1 0 20 Feb 2024
Speech Translation with Speech Foundation Models and Large Language Models: What is There and What is Missing? Marco Gaido Sara Papi Matteo Negri L. Bentivogli 79 13 0 19 Feb 2024
Pushing the Limits of Zero-shot End-to-End Speech Translation Ioannis Tsiamas Gerard I. Gállego José A. R. Fonollosa Marta R. Costa-jussá 48 7 0 16 Feb 2024
Integrating Pre-Trained Speech and Language Models for End-to-End Speech Recognition Yukiya Hono Koh Mitsuda Tianyu Zhao Kentaro Mitsui Toshiaki Wakatsuki Kei Sawada AuLLM 56 8 0 06 Dec 2023
Multi-resolution HuBERT: Multi-resolution Speech Self-Supervised Learning with Masked Unit Prediction Jiatong Shi Hirofumi Inaguma Xutai Ma Ilia Kulikov Anna Y. Sun 52 24 0 04 Oct 2023
Cross-Modal Multi-Tasking for Speech-to-Text Translation via Hard Parameter Sharing B. Grimstad Xuankai Chang Antonios Anastasopoulos Yuya Fujita Shinji Watanabe 53 2 0 27 Sep 2023
Direct Models for Simultaneous Translation and Automatic Subtitling: FBK@IWSLT2023 Sara Papi Marco Gaido Matteo Negri 48 7 0 27 Sep 2023
Decoder-only Architecture for Speech Recognition with CTC Prompts and Text Data Augmentation E. Tsunoo Hayato Futami Yosuke Kashiwagi Siddhant Arora Shinji Watanabe VLM AuLLM RALM 46 9 0 16 Sep 2023
On decoder-only architecture for speech-to-text and large language model integration Jian Wu Yashesh Gaur Zhuo Chen Long Zhou Yilun Zhu ... Jinyu Li Shujie Liu Bo Ren Linquan Liu Yu-Huan Wu AuLLM 41 122 0 08 Jul 2023
Speech Translation with Foundation Models and Optimal Transport: UPC at IWSLT23 Ioannis Tsiamas Gerard I. Gállego José A. R. Fonollosa Marta R. Costa-jussá OT 21 3 0 02 Jun 2023
AlignAtt: Using Attention-based Audio-Translation Alignments as a Guide for Simultaneous Speech Translation Sara Papi Marco Turchi Matteo Negri 40 20 0 19 May 2023
ESPnet-ST-v2: Multipurpose Spoken Language Translation Toolkit Brian Yan Jiatong Shi Yun Tang Hirofumi Inaguma Yifan Peng ... Zhaoheng Ni Moto Hira Soumi Maiti J. Pino Shinji Watanabe 38 20 0 10 Apr 2023
When Good and Reproducible Results are a Giant with Feet of Clay: The Importance of Software Quality in NLP Sara Papi Marco Gaido Andrea Pilzer Matteo Negri 79 10 0 28 Mar 2023
Efficient CTC Regularization via Coarse Labels for End-to-End Speech Translation Biao Zhang Barry Haddow Rico Sennrich 32 3 0 21 Feb 2023
AdaTranS: Adapting with Boundary-based Shrinking for End-to-End Speech Translation Xingshan Zeng Liangyou Li Qun Liu 34 5 0 17 Dec 2022
Attention as a Guide for Simultaneous Speech Translation Sara Papi Matteo Negri Marco Turchi 37 30 0 15 Dec 2022
Efficient Speech Translation with Pre-trained Models Zhaolin Li Jan Niehues 32 2 0 09 Nov 2022
Joint Speech Translation and Named Entity Recognition Marco Gaido Sara Papi Matteo Negri Marco Turchi 44 3 0 21 Oct 2022
$RedApt: An Adaptor for wav2vec 2 Encoding \\ Faster and Smaller Speech Translation without Quality Compromise$ RedApt: An Adaptor for wav2vec 2 Encoding \\ Faster and Smaller Speech Translation without Quality Compromise Jinming Zhao Haomiao Yang Gholamreza Haffari Ehsan Shareghi VLM 33 2 0 16 Oct 2022
CTC Alignments Improve Autoregressive Translation Brian Yan Siddharth Dalmia Yosuke Higuchi Graham Neubig Florian Metze A. Black Shinji Watanabe 54 33 0 11 Oct 2022
Direct Speech Translation for Automatic Subtitling Sara Papi Marco Gaido Alina Karakanta Mauro Cettolo Matteo Negri Marco Turchi 63 11 0 27 Sep 2022
Dodging the Data Bottleneck: Automatic Subtitling with Automatically Segmented ST Corpora Sara Papi Alina Karakanta Matteo Negri Marco Turchi 41 8 0 21 Sep 2022
M-Adapter: Modality Adaptation for End-to-End Speech-to-Text Translation Jinming Zhao Haomiao Yang Ehsan Shareghi Gholamreza Haffari 75 19 0 03 Jul 2022
Revisiting End-to-End Speech-to-Text Translation From Scratch Biao Zhang Barry Haddow Rico Sennrich 37 36 0 09 Jun 2022
Multiformer: A Head-Configurable Transformer-Based Model for Direct Speech Translation Gerard Sant Gerard I. Gállego Belen Alastruey Marta R. Costa-jussá 32 3 0 14 May 2022
Efficient yet Competitive Speech Translation: FBK@IWSLT2022 Marco Gaido Sara Papi Dennis Fucci G. Fiameni Matteo Negri Marco Turchi 38 19 0 05 May 2022
Does Simultaneous Speech Translation need Simultaneous Models? Sara Papi Marco Gaido Matteo Negri Marco Turchi 50 26 0 08 Apr 2022
Fast-MD: Fast Multi-Decoder End-to-End Speech Translation with Non-Autoregressive Hidden Intermediates Hirofumi Inaguma Siddharth Dalmia Brian Yan Shinji Watanabe 65 11 0 27 Sep 2021
Speechformer: Reducing Information Loss in Direct Speech Translation Sara Papi Marco Gaido Matteo Negri Marco Turchi 88 23 0 09 Sep 2021
Non-autoregressive End-to-end Speech Translation with Parallel Autoregressive Rescoring Hirofumi Inaguma Yosuke Higuchi Kevin Duh Tatsuya Kawahara Shinji Watanabe 63 11 0 09 Sep 2021
Zero-shot Speech Translation Tu Anh Dinh 38 6 0 13 Jul 2021
End-to-End Automatic Speech Translation of Audiobooks Alexandre Berard Laurent Besacier A. Kocabiyikoglu Olivier Pietquin 83 190 0 12 Feb 2018