SUTRA: Scalable Multilingual Language Model Architecture

7 May 2024

Papers citing "SUTRA: Scalable Multilingual Language Model Architecture"

30 / 30 papers shown

Title
RakutenAI-7B: Extending Large Language Models for Japanese Rakuten Group Aaron Levine Connie Huang Chenguang Wang Eduardo Batista ... Ting Cai Wei-Te Chen Yandi Xia Yuki Nakayama Yutaka Higashiyama 52 9 0 21 Mar 2024
Chatbot Arena: An Open Platform for Evaluating LLMs by Human Preference Wei-Lin Chiang Lianmin Zheng Ying Sheng Anastasios Nikolas Angelopoulos Tianle Li ... Hao Zhang Banghua Zhu Michael I. Jordan Joseph E. Gonzalez Ion Stoica OSLM 134 569 0 07 Mar 2024
KMMLU: Measuring Massive Multitask Language Understanding in Korean Guijin Son Hanwool Albert Lee Sungdong Kim Seungone Kim Niklas Muennighoff Taekyoon Choi Cheonbok Park Kang Min Yoo Stella Biderman ALM RALM ELM 71 43 0 18 Feb 2024
Aya Model: An Instruction Finetuned Open-Access Multilingual Language Model Ahmet Üstün Viraat Aryabumi Zheng-Xin Yong Wei-Yin Ko Daniel D'souza ... Shayne Longpre Niklas Muennighoff Marzieh Fadaee Julia Kreutzer Sara Hooker ALM ELM SyDa LRM 67 224 0 12 Feb 2024
Airavata: Introducing Hindi Instruction-tuned LLM Jay Gala Thanmay Jayakumar Jaavid Aktar Husain M. AswanthKumar Mohammed Safi Ur Rahman Khan ... Ratish Puduppully Mitesh M. Khapra Raj Dabre Rudra Murthy Anoop Kunchukuttan 67 27 0 26 Jan 2024
Mistral 7B Albert Q. Jiang Alexandre Sablayrolles A. Mensch Chris Bamford Devendra Singh Chaplot ... Teven Le Scao Thibaut Lavril Thomas Wang Timothée Lacroix William El Sayed MoE LRM 65 2,192 0 10 Oct 2023
Jais and Jais-chat: Arabic-Centric Foundation and Instruction-Tuned Open Generative Large Language Models Neha Sengupta Sunil Kumar Sahu Bokang Jia Satheesh Katipomu Haonan Li ... A. Jackson Hector Xuguang Ren Preslav Nakov Timothy Baldwin Eric P. Xing LRM 66 41 0 30 Aug 2023
Okapi: Instruction-tuned Large Language Models in Multiple Languages with Reinforcement Learning from Human Feedback Viet Dac Lai Chien Van Nguyen Nghia Trung Ngo Thuat Nguyen Franck Dernoncourt Ryan Rossi Thien Huu Nguyen ALM 80 149 0 29 Jul 2023
Llama 2: Open Foundation and Fine-Tuned Chat Models Hugo Touvron Louis Martin Kevin R. Stone Peter Albert Amjad Almahairi ... Sharan Narang Aurelien Rodriguez Robert Stojnic Sergey Edunov Thomas Scialom AI4MH ALM 273 11,828 0 18 Jul 2023
LLM-powered Data Augmentation for Enhanced Cross-lingual Performance Chenxi Whitehouse Monojit Choudhury Alham Fikri Aji SyDa LRM 60 74 0 23 May 2023
OpenAssistant Conversations -- Democratizing Large Language Model Alignment Andreas Kopf Yannic Kilcher Dimitri von Rutte Sotiris Anagnostidis Zhi Rui Tam ... Arnav Dantuluri Andrew Maguire Christoph Schuhmann Huu Nguyen A. Mattick ALM LM&MA 119 628 0 14 Apr 2023
Measuring and Narrowing the Compositionality Gap in Language Models Ofir Press Muru Zhang Sewon Min Ludwig Schmidt Noah A. Smith M. Lewis ReLM KELM LRM 152 624 0 07 Oct 2022
No Language Left Behind: Scaling Human-Centered Machine Translation Nllb team Marta R. Costa-jussá James Cross Onur cCelebi Maha Elbayad ... Alexandre Mourachko C. Ropers Safiyyah Saleem Holger Schwenk Jeff Wang MoE 215 1,258 0 11 Jul 2022
OPT: Open Pre-trained Transformer Language Models Susan Zhang Stephen Roller Naman Goyal Mikel Artetxe Moya Chen ... Daniel Simig Punit Singh Koura Anjali Sridhar Tianlu Wang Luke Zettlemoyer VLM OSLM AI4CE 299 3,647 0 02 May 2022
Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedback Yuntao Bai Andy Jones Kamal Ndousse Amanda Askell Anna Chen ... Jack Clark Sam McCandlish C. Olah Benjamin Mann Jared Kaplan 239 2,546 0 12 Apr 2022
Mixture-of-Experts with Expert Choice Routing Yan-Quan Zhou Tao Lei Han-Chu Liu Nan Du Yanping Huang Vincent Zhao Andrew M. Dai Zhifeng Chen Quoc V. Le James Laudon MoE 279 355 0 18 Feb 2022
Unified Scaling Laws for Routed Language Models Aidan Clark Diego de Las Casas Aurelia Guy A. Mensch Michela Paganini ... Oriol Vinyals Jack W. Rae Erich Elsen Koray Kavukcuoglu Karen Simonyan MoE 61 182 0 02 Feb 2022
Internet-Augmented Dialogue Generation M. Komeili Kurt Shuster Jason Weston RALM 289 287 0 15 Jul 2021
DSelect-k: Differentiable Selection in the Mixture of Experts with Applications to Multi-Task Learning Hussein Hazimeh Zhe Zhao Aakanksha Chowdhery M. Sathiamoorthy Yihua Chen Rahul Mazumder Lichan Hong Ed H. Chi MoE 140 144 0 07 Jun 2021
Improving Multilingual Models with Language-Clustered Vocabularies Hyung Won Chung Dan Garrette Kiat Chuan Tan Jason Riesa VLM 102 65 0 24 Oct 2020
Measuring Massive Multitask Language Understanding Dan Hendrycks Collin Burns Steven Basart Andy Zou Mantas Mazeika D. Song Jacob Steinhardt ELM RALM 164 4,413 0 07 Sep 2020
Language Models are Few-Shot Learners Tom B. Brown Benjamin Mann Nick Ryder Melanie Subbiah Jared Kaplan ... Christopher Berner Sam McCandlish Alec Radford Ilya Sutskever Dario Amodei BDL 731 41,894 0 28 May 2020
Unsupervised Cross-lingual Representation Learning at Scale Alexis Conneau Kartikay Khandelwal Naman Goyal Vishrav Chaudhary Guillaume Wenzek Francisco Guzmán Edouard Grave Myle Ott Luke Zettlemoyer Veselin Stoyanov 199 6,546 0 05 Nov 2019
ChID: A Large-scale Chinese IDiom Dataset for Cloze Test Chujie Zheng Minlie Huang Aixin Sun 59 87 0 04 Jun 2019
Transformer-XL: Attentive Language Models Beyond a Fixed-Length Context Zihang Dai Zhilin Yang Yiming Yang J. Carbonell Quoc V. Le Ruslan Salakhutdinov VLM 214 3,726 0 09 Jan 2019
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding Jacob Devlin Ming-Wei Chang Kenton Lee Kristina Toutanova VLM SSL SSeg 1.7K 94,729 0 11 Oct 2018
Six Challenges for Neural Machine Translation Philipp Koehn Rebecca Knowles AAML AIMat 366 1,224 0 12 Jun 2017
Attention Is All You Need Ashish Vaswani Noam M. Shazeer Niki Parmar Jakob Uszkoreit Llion Jones Aidan Gomez Lukasz Kaiser Illia Polosukhin 3DV 670 131,414 0 12 Jun 2017
Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layer Noam M. Shazeer Azalia Mirhoseini Krzysztof Maziarz Andy Davis Quoc V. Le Geoffrey E. Hinton J. Dean MoE 246 2,643 0 23 Jan 2017
Google's Neural Machine Translation System: Bridging the Gap between Human and Machine Translation Yonghui Wu M. Schuster Zhiwen Chen Quoc V. Le Mohammad Norouzi ... Alex Rudnick Oriol Vinyals G. Corrado Macduff Hughes J. Dean AIMat 891 6,787 0 26 Sep 2016