Composable Sparse Fine-Tuning for Cross-Lingual Transfer

14 October 2021

Papers citing "Composable Sparse Fine-Tuning for Cross-Lingual Transfer"

43 / 43 papers shown

Title
Communication-Efficient Federated Fine-Tuning of Language Models via Dynamic Update Schedules Michail Theologitis V. Samoladas Antonios Deligiannakis 36 0 0 07 May 2025
NEAT: Nonlinear Parameter-efficient Adaptation of Pre-trained Models Yibo Zhong Haoxiang Jiang Lincan Li Ryumei Nakada Tianci Liu Linjun Zhang Huaxiu Yao Haoyu Wang 85 2 0 24 Feb 2025
Sparsity May Be All You Need: Sparse Random Parameter Adaptation Jesus Rios Pierre Dognin Ronny Luss Karthikeyan N. Ramamurthy 32 1 0 21 Feb 2025
An Efficient Row-Based Sparse Fine-Tuning Cen-Jhih Li Aditya Bhaskara 58 0 0 17 Feb 2025
Sparse High Rank Adapters K. Bhardwaj N. Pandey Sweta Priyadarshi Viswanath Ganapathy Rafael Esteves ... P. Whatmough Risheek Garrepalli M. V. Baalen Harris Teague Markus Nagel MQ 43 4 0 28 Jan 2025
BLoB: Bayesian Low-Rank Adaptation by Backpropagation for Large Language Models Yibin Wang Haizhou Shi Ligong Han Dimitris N. Metaxas Hao Wang BDL UQLM 116 8 0 28 Jan 2025
Parameter-Efficient Fine-Tuning in Large Models: A Survey of Methodologies Liwen Wang Sheng Chen Linnan Jiang Shu Pan Runze Cai Sen Yang Fei Yang 52 3 0 24 Oct 2024
Understanding Layer Significance in LLM Alignment Guangyuan Shi Zexin Lu Xiaoyu Dong Wenlong Zhang Xuanyu Zhang Yujie Feng Xiao-Ming Wu 58 2 0 23 Oct 2024
Layer Swapping for Zero-Shot Cross-Lingual Transfer in Large Language Models Lucas Bandarkar Benjamin Muller Pritish Yuvraj Rui Hou Nayan Singhal Hongjiang Lv Bing-Quan Liu KELM LRM MoMe 52 3 0 02 Oct 2024
Circuit Compositions: Exploring Modular Structures in Transformer-Based Language Models Philipp Mondorf Sondre Wold Yun Xue 41 0 0 02 Oct 2024
Comparing LLM prompting with Cross-lingual transfer performance on Indigenous and Low-resource Brazilian Languages David Ifeoluwa Adelani A. S. Dougruoz André Coneglian Atul Kr. Ojha 39 2 0 28 Apr 2024
No Train but Gain: Language Arithmetic for training-free Language Adapters enhancement Mateusz Klimaszewski Piotr Andruszkiewicz Alexandra Birch MoMe 47 4 0 24 Apr 2024
PiSSA: Principal Singular Values and Singular Vectors Adaptation of Large Language Models Fanxu Meng Zhaohui Wang Muhan Zhang VLM 64 76 0 03 Apr 2024
Hazards from Increasingly Accessible Fine-Tuning of Downloadable Foundation Models Alan Chan Ben Bucknall Herbie Bradley David M. Krueger 16 6 0 22 Dec 2023
Language and Task Arithmetic with Parameter-Efficient Layers for Zero-Shot Summarization Alexandra Chronopoulou Jonas Pfeiffer Joshua Maynez Xinyi Wang Sebastian Ruder Priyanka Agrawal MoMe 26 16 0 15 Nov 2023
Model Merging by Uncertainty-Based Gradient Matching Nico Daheim Thomas Möllenhoff Edoardo Ponti Iryna Gurevych Mohammad Emtiyaz Khan MoMe FedML 34 45 0 19 Oct 2023
IncreLoRA: Incremental Parameter Allocation Method for Parameter-Efficient Fine-tuning Feiyu F. Zhang Liangzhi Li Jun-Cheng Chen Zhouqian Jiang Bowen Wang Yiming Qian 51 33 0 23 Aug 2023
Improving Language Plasticity via Pretraining with Active Forgetting Yihong Chen Kelly Marchisio Roberta Raileanu David Ifeoluwa Adelani Pontus Stenetorp Sebastian Riedel Mikel Artetx KELM AI4CE CLL 37 24 0 03 Jul 2023
Cross-Lingual Transfer with Target Language-Ready Task Adapters Marinela Parović Alan Ansell Ivan Vulić Anna Korhonen 40 9 0 05 Jun 2023
A Framework For Refining Text Classification and Object Recognition from Academic Articles Jinghong Li Koichi Ota Wen Gu S. Hasegawa 38 0 0 27 May 2023
Parameter-Efficient Fine-Tuning without Introducing New Latency Baohao Liao Yan Meng Christof Monz 24 49 0 26 May 2023
MasakhaPOS: Part-of-Speech Tagging for Typologically Diverse African Languages Cheikh M. Bamba Dione David Ifeoluwa Adelani Peter Nabende Jesujoba Oluwadara Alabi Thapelo Sindane ... Seydou T. Traoré C. Uchechukwu Aliyu Yusuf M. Abdullahi Dietrich Klakow 27 13 0 23 May 2023
TADA: Efficient Task-Agnostic Domain Adaptation for Transformers Chia-Chien Hung Lukas Lange Jannik Strötgen 30 9 0 22 May 2023
Glot500: Scaling Multilingual Corpora and Language Models to 500 Languages Ayyoob Imani Peiqin Lin Amir Hossein Kargaran Silvia Severini Masoud Jalili Sabet ... Chunlan Ma Helmut Schmid André F. T. Martins François Yvon Hinrich Schütze ALM LRM 49 96 0 20 May 2023
Elastic Weight Removal for Faithful and Abstractive Dialogue Generation Nico Daheim Nouha Dziri Mrinmaya Sachan Iryna Gurevych Edoardo Ponti MoMe 39 30 0 30 Mar 2023
Towards Zero-Shot Functional Compositionality of Language Models Hangyeol Yu Myeongho Jeong Jamin Shin Hyeongdon Moon Juneyoung Park Seungtaek Choi 42 1 0 06 Mar 2023
Modular Deep Learning Jonas Pfeiffer Sebastian Ruder Ivan Vulić Edoardo Ponti MoMe OOD 34 73 0 22 Feb 2023
AutoPEFT: Automatic Configuration Search for Parameter-Efficient Fine-Tuning Han Zhou Xingchen Wan Ivan Vulić Anna Korhonen 24 45 0 28 Jan 2023
BLOOM+1: Adding Language Support to BLOOM for Zero-Shot Prompting Zheng-Xin Yong Hailey Schoelkopf Niklas Muennighoff Alham Fikri Aji David Ifeoluwa Adelani ... Genta Indra Winata Stella Biderman Edward Raff Dragomir R. Radev Vassilina Nikoulina CLL VLM AI4CE LRM 35 81 0 19 Dec 2022
Intriguing Properties of Compression on Multilingual Models Kelechi Ogueji Orevaoghene Ahia Gbemileke Onilude Sebastian Gehrmann Sara Hooker Julia Kreutzer 26 12 0 04 Nov 2022
CTC Alignments Improve Autoregressive Translation Brian Yan Siddharth Dalmia Yosuke Higuchi Graham Neubig Florian Metze A. Black Shinji Watanabe 46 33 0 11 Oct 2022
Parameter-Efficient Finetuning for Robust Continual Multilingual Learning Kartikeya Badola Shachi Dave Partha P. Talukdar CLL KELM 39 7 0 14 Sep 2022
Improving the Cross-Lingual Generalisation in Visual Question Answering Farhad Nooralahzadeh Rico Sennrich 35 5 0 07 Sep 2022
Lifting the Curse of Multilinguality by Pre-training Modular Transformers Jonas Pfeiffer Naman Goyal Xi Lin Xian Li James Cross Sebastian Riedel Mikel Artetxe LRM 40 139 0 12 May 2022
Parameter-Efficient Neural Reranking for Cross-Lingual and Multilingual Retrieval Robert Litschko Ivan Vulić Goran Glavaš LRM 42 14 0 05 Apr 2022
Combining Modular Skills in Multitask Learning Edoardo Ponti Alessandro Sordoni Yoshua Bengio Siva Reddy MoE 17 37 0 28 Feb 2022
IGLUE: A Benchmark for Transfer Learning across Modalities, Tasks, and Languages Emanuele Bugliarello Fangyu Liu Jonas Pfeiffer Siva Reddy Desmond Elliott Edoardo Ponti Ivan Vulić MLLM VLM ELM 50 62 0 27 Jan 2022
Raise a Child in Large Language Model: Towards Effective and Generalizable Fine-tuning Runxin Xu Fuli Luo Zhiyuan Zhang Chuanqi Tan Baobao Chang Songfang Huang Fei Huang LRM 151 178 0 13 Sep 2021
AmericasNLI: Evaluating Zero-shot Natural Language Understanding of Pretrained Multilingual Models in Truly Low-resource Languages Abteen Ebrahimi Manuel Mager Arturo Oncevay Vishrav Chaudhary Luis Chiruzzo ... Graham Neubig Alexis Palmer Rolando A. Coto Solano Ngoc Thang Vu Katharina Kann 111 72 0 18 Apr 2021
Orthogonal Language and Task Adapters in Zero-Shot Cross-Lingual Transfer M. Vidoni Ivan Vulić Goran Glavaš 33 27 0 11 Dec 2020
Comparing Rewinding and Fine-tuning in Neural Network Pruning Alex Renda Jonathan Frankle Michael Carbin 235 383 0 05 Mar 2020
MLQA: Evaluating Cross-lingual Extractive Question Answering Patrick Lewis Barlas Oğuz Ruty Rinott Sebastian Riedel Holger Schwenk ELM 246 495 0 16 Oct 2019
GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding Alex Jinpeng Wang Amanpreet Singh Julian Michael Felix Hill Omer Levy Samuel R. Bowman ELM 304 6,996 0 20 Apr 2018