34

Tiny Aya: Bridging Scale and Multilingual Depth

Alejandro R. Salamanca
Diana Abagyan
Daniel D'souza
Ammar Khairi
David Mora
Saurabh Dash
Viraat Aryabumi
Sara Rajaee
Mehrnaz Mofakhami
Ananya Sahu
Thomas Euyang
Brittawnya Prince
Madeline Smith
Hangyu Lin
Acyr Locatelli
Sara Hooker
Tom Kocmi
Aidan Gomez
Ivan Zhang
Phil Blunsom
Nick Frosst
Joelle Pineau
Beyza Ermis
Ahmet Üstün
Julia Kreutzer
Marzieh Fadaee
Main:30 Pages
15 Figures
Bibliography:7 Pages
32 Tables
Appendix:13 Pages
Abstract

Tiny Aya redefines what a small multilingual language model can achieve. Trained on 70 languages and refined through region-aware posttraining, it delivers state-of-the-art in translation quality, strong multilingual understanding, and high-quality target-language generation, all with just 3.35B parameters. The release includes a pretrained foundation model, a globally balanced instruction-tuned variant, and three region-specialized models targeting languages from Africa, South Asia, Europe, Asia-Pacific, and West Asia. This report details the training strategy, data composition, and comprehensive evaluation framework behind Tiny Aya, and presents an alternative scaling path for multilingual AI: one centered on efficiency, balanced performance across languages, and practical deployment.

View on arXiv
Comments on this paper