Adapting Pre-trained Language Models to African Languages via Multilingual Adaptive Fine-Tuning

13 April 2022

Jesujoba Oluwadara Alabi

David Ifeoluwa Adelani

Marius Mosbach

Dietrich Klakow

ArXiv PDF HTML

Papers citing "Adapting Pre-trained Language Models to African Languages via Multilingual Adaptive Fine-Tuning"

31 / 31 papers shown

Title
HYPEROFA: Expanding LLM Vocabulary to New Languages via Hypernetwork-Based Embedding Initialization Enes Özeren Yihong Liu Hinrich Schütze 31 0 0 21 Apr 2025
LAG-MMLU: Benchmarking Frontier LLM Understanding in Latvian and Giriama Naome A. Etori Kevin Lu Randu Karisa Arturs Kanepajs LRM ELM 160 0 0 14 Mar 2025
Extracting General-use Transformers for Low-resource Languages via Knowledge Distillation Jan Christian Blaise Cruz Alham Fikri Aji 43 1 0 22 Jan 2025
From Tokens to Words: On the Inner Lexicon of LLMs Guy Kaplan Matanel Oren Yuval Reif Roy Schwartz 48 12 0 08 Oct 2024
SSP: Self-Supervised Prompting for Cross-Lingual Transfer to Low-Resource Languages using Large Language Models Vipul Rathore Aniruddha Deb Ankish Chandresh Parag Singla Mausam LRM 52 0 0 27 Jun 2024
UniBridge: A Unified Approach to Cross-Lingual Transfer Learning for Low-Resource Languages Trinh Pham Khoi M. Le Luu Anh Tuan 42 1 0 14 Jun 2024
IrokoBench: A New Benchmark for African Languages in the Age of Large Language Models David Ifeoluwa Adelani Jessica Ojo Israel Abebe Azime Jian Yun Zhuang Jesujoba Oluwadara Alabi ... Salomey Osei Sokhar Samb Tadesse Kebede Guge Pontus Stenetorp Pontus Stenetorp ELM 65 7 0 05 Jun 2024
Chasing COMET: Leveraging Minimum Bayes Risk Decoding for Self-Improving Machine Translation Kamil Guttmann Miko Pokrywka Adrian Charkiewicz Artur Nowakowski 58 3 0 20 May 2024
Comparing LLM prompting with Cross-lingual transfer performance on Indigenous and Low-resource Brazilian Languages David Ifeoluwa Adelani A. S. Dougruoz André Coneglian Atul Kr. Ojha 34 2 0 28 Apr 2024
ANGOFA: Leveraging OFA Embedding Initialization and Synthetic Data for Angolan Language Model Osvaldo Luamba Quinjica David Ifeoluwa Adelani 35 0 0 03 Apr 2024
SemEval-2024 Task 1: Semantic Textual Relatedness for African and Asian Languages N. Ousidhoum Shamsuddeen Hassan Muhammad Mohamed Abdalla Idris Abdulmumin I. Ahmad ... Thamar Solorio Nirmal Surange Krishnapriya Vishnubhotla Seid Muhie Yimam Saif M. Mohammad 44 11 0 27 Mar 2024
LinguAlchemy: Fusing Typological and Geographical Elements for Unseen Language Generalization Muhammad Farid Adilazuarda Samuel Cahyawijaya Alham Fikri Aji Genta Indra Winata Ayu Purwarianti 21 5 0 11 Jan 2024
When Is Multilinguality a Curse? Language Modeling for 250 High- and Low-Resource Languages Tyler A. Chang Catherine Arnett Zhuowen Tu Benjamin Bergen LRM 43 7 0 15 Nov 2023
GradSim: Gradient-Based Language Grouping for Effective Multilingual Training Mingyang Wang Heike Adel Lukas Lange Jannik Strötgen Hinrich Schütze 30 3 0 23 Oct 2023
Lexicon and Rule-based Word Lemmatization Approach for the Somali Language Shafie Abdi Mohamed Muhidin A. Mohamed 12 2 0 03 Aug 2023
Improving Language Plasticity via Pretraining with Active Forgetting Yihong Chen Kelly Marchisio Roberta Raileanu David Ifeoluwa Adelani Pontus Stenetorp Sebastian Riedel Mikel Artetx KELM AI4CE CLL 30 23 0 03 Jul 2023
Low-Resource Cross-Lingual Adaptive Training for Nigerian Pidgin Pin-Jie Lin Muhammed Saeed Ernie Chang Merel C. J. Scholman 40 5 0 01 Jul 2023
MasakhaPOS: Part-of-Speech Tagging for Typologically Diverse African Languages Cheikh M. Bamba Dione David Ifeoluwa Adelani Peter Nabende Jesujoba Oluwadara Alabi Thapelo Sindane ... Seydou T. Traoré C. Uchechukwu Aliyu Yusuf M. Abdullahi Dietrich Klakow 24 13 0 23 May 2023
Glot500: Scaling Multilingual Corpora and Language Models to 500 Languages Ayyoob Imani Peiqin Lin Amir Hossein Kargaran Silvia Severini Masoud Jalili Sabet ... Chunlan Ma Helmut Schmid André F. T. Martins François Yvon Hinrich Schütze ALM LRM 33 95 0 20 May 2023
Taxi1500: A Multilingual Dataset for Text Classification in 1500 Languages Chunlan Ma Ayyoob Imani Haotian Ye Renhao Pei Ehsaneddin Asgari Hinrich Schütze 27 23 0 15 May 2023
How Good are Commercial Large Language Models on African Languages? Jessica Ojo Kelechi Ogueji 26 5 0 11 May 2023
UIO at SemEval-2023 Task 12: Multilingual fine-tuning for sentiment classification in low-resource languages Egil Rønningstad 25 5 0 27 Apr 2023
GMNLP at SemEval-2023 Task 12: Sentiment Analysis with Phylogeny-Based Adapters Md Mahfuz Ibn Alam Ruoyu Xie Fahim Faisal Antonios Anastasopoulos 32 3 0 25 Apr 2023
UBC-DLNLP at SemEval-2023 Task 12: Impact of Transfer Learning on African Sentiment Analysis Gagan Bhatia Ife Adebara AbdelRahim Elmadany Muhammad Abdul-Mageed 29 1 0 21 Apr 2023
MasakhaNEWS: News Topic Classification for African languages David Ifeoluwa Adelani Marek Masiak Israel Abebe Azime Jesujoba Oluwadara Alabi A. Tonja ... Moges Ahmed Mehamed Evrard Ngabire Jules Jules Ivan Ssenkungu Pontus Stenetorp 25 23 0 19 Apr 2023
Masakhane-Afrisenti at SemEval-2023 Task 12: Sentiment Analysis using Afro-centric Language Models and Adapters for Low-resource African Languages Israel Abebe Azime Sana Al-Azzawi A. Tonja Iyanuoluwa Shode Jesujoba Oluwadara Alabi ... Mardiyyah Oduwole Tosin P. Adewumi Samuel Fanijo Oyinkansola F. Awosan Oreen Yousuf 22 1 0 13 Apr 2023
SwissBERT: The Multilingual Language Model for Switzerland Jannis Vamvas Johannes Graen Rico Sennrich 38 6 0 23 Mar 2023
Separating Grains from the Chaff: Using Data Filtering to Improve Multilingual Translation for Low-Resourced African Languages Idris Abdulmumin Michael Beukman Jesujoba Oluwadara Alabi Chris C. Emezue Everlyn Asiko ... Shamsuddeen Hassan Muhammad Mofetoluwa Adeyemi Oreen Yousuf Sahib Singh T. Gwadabe 34 6 0 19 Oct 2022
ERNIE-M: Enhanced Multilingual Representation by Aligning Cross-lingual Semantics with Monolingual Corpora Ouyang Xuan Shuohuan Wang Chao Pang Yu Sun Hao Tian Hua-Hong Wu Haifeng Wang 62 100 0 31 Dec 2020
When Being Unseen from mBERT is just the Beginning: Handling New Languages With Multilingual Language Models Benjamin Muller Antonis Anastasopoulos Benoît Sagot Djamé Seddah LRM 134 165 0 24 Oct 2020
Rethinking embedding coupling in pre-trained language models Hyung Won Chung Thibault Févry Henry Tsai Melvin Johnson Sebastian Ruder 95 142 0 24 Oct 2020