How Effective are State Space Models for Machine Translation?

How Effective are State Space Models for Machine Translation?

7 July 2024

Pavlo Vasylenko

Marcos Vinícius Treviso

André F. T. Martins

ArXiv (abs)PDF HTML

Papers citing "How Effective are State Space Models for Machine Translation?"

16 / 16 papers shown

Title
Systematic Generalization in Language Models Scales with Information Entropy Sondre Wold Lucas Georges Gabriel Charpentier Étienne Simon 203 0 0 19 May 2025
Zamba: A Compact 7B SSM Hybrid Model Paolo Glorioso Quentin G. Anthony Yury Tokpanov James Whittington Jonathan Pilault Adam Ibrahim Beren Millidge 80 49 0 26 May 2024
Gated Linear Attention Transformers with Hardware-Efficient Training Aaron Courville Bailin Wang Songlin Yang Yikang Shen Yoon Kim 118 180 0 11 Dec 2023
Never Train from Scratch: Fair Comparison of Long-Sequence Models Requires Data-Driven Priors Ido Amos Jonathan Berant Ankit Gupta 98 28 0 04 Oct 2023
RWKV: Reinventing RNNs for the Transformer Era Bo Peng Eric Alcaide Quentin G. Anthony Alon Albalak Samuel Arcadinho ... Qihang Zhao P. Zhou Qinghua Zhou Jian Zhu Rui-Jie Zhu 235 609 0 22 May 2023
Document-Level Machine Translation with Large Language Models Longyue Wang Chenyang Lyu Tianbo Ji Zhirui Zhang Dian Yu Shuming Shi Zhaopeng Tu ELM 85 125 0 05 Apr 2023
Detecting and Mitigating Hallucinations in Machine Translation: Model Internal Workings Alone Do Well, Sentence Similarity Even Better David Dale Elena Voita Loïc Barrault Marta R. Costa-jussá HILM 218 73 0 16 Dec 2022
RoFormer: Enhanced Transformer with Rotary Position Embedding Jianlin Su Yu Lu Shengfeng Pan Ahmed Murtadha Bo Wen Yunfeng Liu 327 2,533 0 20 Apr 2021
The Pile: An 800GB Dataset of Diverse Text for Language Modeling Leo Gao Stella Biderman Sid Black Laurence Golding Travis Hoppe ... Horace He Anish Thite Noa Nabeshima Shawn Presser Connor Leahy AIMat 476 2,123 0 31 Dec 2020
HiPPO: Recurrent Memory with Optimal Polynomial Projections Albert Gu Tri Dao Stefano Ermon Atri Rudra Christopher Ré 129 535 0 17 Aug 2020
Transformers are RNNs: Fast Autoregressive Transformers with Linear Attention Angelos Katharopoulos Apoorv Vyas Nikolaos Pappas Franccois Fleuret 209 1,793 0 29 Jun 2020
Longformer: The Long-Document Transformer Iz Beltagy Matthew E. Peters Arman Cohan RALM VLM 185 4,100 0 10 Apr 2020
Transformer Dissection: A Unified Understanding of Transformer's Attention via the Lens of Kernel Yao-Hung Hubert Tsai Shaojie Bai M. Yamada Louis-Philippe Morency Ruslan Salakhutdinov 129 261 0 30 Aug 2019
Generating Long Sequences with Sparse Transformers R. Child Scott Gray Alec Radford Ilya Sutskever 135 1,919 0 23 Apr 2019
A Call for Clarity in Reporting BLEU Scores Matt Post 184 2,998 0 23 Apr 2018
Neural Machine Translation of Rare Words with Subword Units Rico Sennrich Barry Haddow Alexandra Birch 238 7,760 0 31 Aug 2015