v1v2v3v4 (latest)

Fine-tuned Language Models are Continual Learners

24 May 2022

Smaranda Muresan

Papers citing "Fine-tuned Language Models are Continual Learners"

42 / 42 papers shown

Title
A Modular Approach for Clinical SLMs Driven by Synthetic Data with Pre-Instruction Tuning, Model Merging, and Clinical-Tasks Alignment Jean-Philippe Corbeil Amin Dada Jean-Michel Attendu Asma Ben Abacha Alessandro Sordoni Lucas Caccia François Beaulieu Thomas Lin Jens Kleesiek Paul Vozila LM&MA 102 0 0 15 May 2025
Unlocking the Power of Function Vectors for Characterizing and Mitigating Catastrophic Forgetting in Continual Instruction Tuning Gangwei Jiang Caigao Jiang Zhaoyi Li Siqiao Xue Jun-ping Zhou Linqi Song Defu Lian Yin Wei CLL MU 146 2 0 16 Feb 2025
Assessing Open-world Forgetting in Generative Image Model Customization Héctor Laria Alex Gomez-Villa Imad Eddine Marouf Bogdan Raducanu Bogdan Raducanu VLM DiffM 89 0 0 18 Oct 2024
Algorithmic Behaviors Across Regions: A Geolocation Audit of YouTube Search for COVID-19 Misinformation Between the United States and South Africa Hayoung Jung Prerna Juneja Tanushree Mitra MLAU 113 1 0 16 Sep 2024
Towards Understanding Multi-Task Learning (Generalization) of LLMs via Detecting and Exploring Task-Specific Neurons Yongqi Leng Deyi Xiong 93 8 0 09 Jul 2024
The BiGGen Bench: A Principled Benchmark for Fine-grained Evaluation of Language Models with Language Models Seungone Kim Juyoung Suk Ji Yong Cho Shayne Longpre Chaeeun Kim ... Sean Welleck Graham Neubig Moontae Lee Kyungjae Lee Minjoon Seo ELM ALM LM&MA 183 44 0 09 Jun 2024
CorDA: Context-Oriented Decomposition Adaptation of Large Language Models for Task-Aware Parameter-Efficient Fine-tuning Yibo Yang Xiaojie Li Zhongzhu Zhou Shuaiwen Leon Song Jianlong Wu Liqiang Nie Guohao Li 77 14 0 07 Jun 2024
Don't Half-listen: Capturing Key-part Information in Continual Instruction Tuning Yongquan He Wenyuan Zhang Xuancheng Huang Peng Zhang Lingxun Meng Wei Lin Wenyuan Zhang Yifu Gao CLL ALM 128 5 0 15 Mar 2024
Investigating Continual Pretraining in Large Language Models: Insights and Implications cCaugatay Yildiz Nishaanth Kanna Ravichandran Prishruit Punia Matthias Bethge Beyza Ermis CLL KELM LRM 102 30 0 27 Feb 2024
An Empirical Study of Catastrophic Forgetting in Large Language Models During Continual Fine-tuning Yun Luo Zhen Yang Fandong Meng Yafu Li Jie Zhou Yue Zhang CLL KELM 184 315 0 17 Aug 2023
CLiMB: A Continual Learning Benchmark for Vision-and-Language Tasks Tejas Srinivasan Ting-Yun Chang Leticia Pinto-Alva Georgios Chochlakis Mohammad Rostami Jesse Thomason VLM CLL 89 76 0 18 Jun 2022
Continual Pre-Training Mitigates Forgetting in Language and Vision Andrea Cossu Tinne Tuytelaars Antonio Carta Lucia C. Passaro Vincenzo Lomonaco D. Bacciu KELM VLM CLL 69 72 0 19 May 2022
On Continual Model Refinement in Out-of-Distribution Data Streams Bill Yuchen Lin Sida I. Wang Xi Lin Robin Jia Lin Xiao Xiang Ren Wen-tau Yih CLL 68 31 0 04 May 2022
ConTinTin: Continual Learning from Task Instructions Wenpeng Yin Jia Li Caiming Xiong CLL 82 30 0 16 Mar 2022
Using DeepSpeed and Megatron to Train Megatron-Turing NLG 530B, A Large-Scale Generative Language Model Shaden Smith M. Patwary Brandon Norick P. LeGresley Samyam Rajbhandari ... Mohammad Shoeybi Yuxiong He Michael Houston Saurabh Tiwary Bryan Catanzaro MoE 155 742 0 28 Jan 2022
An Empirical Investigation of the Role of Pre-training in Lifelong Learning Sanket Vaibhav Mehta Darshan Patil Sarath Chandar Emma Strubell CLL 119 144 0 16 Dec 2021
Scaling Language Models: Methods, Analysis & Insights from Training Gopher Jack W. Rae Sebastian Borgeaud Trevor Cai Katie Millican Jordan Hoffmann ... Jeff Stanway L. Bennett Demis Hassabis Koray Kavukcuoglu G. Irving 136 1,323 0 08 Dec 2021
Adapting BERT for Continual Learning of a Sequence of Aspect Sentiment Classification Tasks Zixuan Ke Hu Xu Bing-Quan Liu CLL 292 85 0 06 Dec 2021
DyTox: Transformers for Continual Learning with DYnamic TOken eXpansion Arthur Douillard Alexandre Ramé Guillaume Couairon Matthieu Cord CLL 108 313 0 22 Nov 2021
Lifelong Pretraining: Continually Adapting Language Models to Emerging Corpora Xisen Jin Dejiao Zhang Henghui Zhu Wei Xiao Shang-Wen Li Xiaokai Wei Andrew O. Arnold Xiang Ren KELM CLL 93 116 0 16 Oct 2021
Multitask Prompted Training Enables Zero-Shot Task Generalization Victor Sanh Albert Webson Colin Raffel Stephen H. Bach Lintang Sutawika ... T. Bers Stella Biderman Leo Gao Thomas Wolf Alexander M. Rush LRM 355 1,708 0 15 Oct 2021
Finetuned Language Models Are Zero-Shot Learners Jason W. Wei Maarten Bosma Vincent Zhao Kelvin Guu Adams Wei Yu Brian Lester Nan Du Andrew M. Dai Quoc V. Le ALM UQCV 230 3,782 0 03 Sep 2021
Cross-Task Generalization via Natural Language Crowdsourcing Instructions Swaroop Mishra Daniel Khashabi Chitta Baral Hannaneh Hajishirzi LRM 168 752 0 18 Apr 2021
Continual Learning in Task-Oriented Dialogue Systems Andrea Madotto Zhaojiang Lin Zhenpeng Zhou Seungwhan Moon Paul A. Crook Bing-Quan Liu Zhou Yu Eunjoon Cho Zhiguang Wang CLL 128 132 0 31 Dec 2020
Continual Lifelong Learning in Natural Language Processing: A Survey Magdalena Biesialska Katarzyna Biesialska Marta R. Costa-jussá KELM CLL 86 220 0 17 Dec 2020
Synthetic Data Augmentation for Zero-Shot Cross-Lingual Question Answering Arij Riabi Thomas Scialom Rachel Keraron Benoît Sagot Djamé Seddah Jacopo Staiano 191 53 0 23 Oct 2020
Continual Learning for Natural Language Generation in Task-oriented Dialog Systems Fei Mi Liangwei Chen Mengjie Zhao Minlie Huang Boi Faltings CLL KELM 50 71 0 02 Oct 2020
Language Models are Few-Shot Learners Tom B. Brown Benjamin Mann Nick Ryder Melanie Subbiah Jared Kaplan ... Christopher Berner Sam McCandlish Alec Radford Ilya Sutskever Dario Amodei BDL 880 42,379 0 28 May 2020
Neural CRF Model for Sentence Alignment in Text Simplification Chao Jiang Mounica Maddela Wuwei Lan Yang Zhong Wenyuan Xu 81 162 0 05 May 2020
ASSET: A Dataset for Tuning and Evaluation of Sentence Simplification Models with Multiple Rewriting Transformations Fernando Alva-Manchego Louis Martin Antoine Bordes Carolina Scarton Benoît Sagot Lucia Specia 56 144 0 01 May 2020
Ask to Learn: A Study on Curiosity-driven Question Generation Thomas Scialom Jacopo Staiano 60 24 0 08 Nov 2019
Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer Colin Raffel Noam M. Shazeer Adam Roberts Katherine Lee Sharan Narang Michael Matena Yanqi Zhou Wei Li Peter J. Liu AIMat 485 20,317 0 23 Oct 2019
LAMOL: LAnguage MOdeling for Lifelong Language Learning Fan-Keng Sun Cheng-Hao Ho Hung-yi Lee CLL KELM 90 211 0 07 Sep 2019
ELI5: Long Form Question Answering Angela Fan Yacine Jernite Ethan Perez David Grangier Jason Weston Michael Auli AI4MH ELM 106 624 0 22 Jul 2019
Episodic Memory in Lifelong Language Learning Cyprien de Masson dÁutume Sebastian Ruder Lingpeng Kong Dani Yogatama CLL KELM 134 291 0 03 Jun 2019
BERTScore: Evaluating Text Generation with BERT Tianyi Zhang Varsha Kishore Felix Wu Kilian Q. Weinberger Yoav Artzi 352 5,868 0 21 Apr 2019
e-SNLI: Natural Language Inference with Natural Language Explanations Oana-Maria Camburu Tim Rocktaschel Thomas Lukasiewicz Phil Blunsom LRM 425 640 0 04 Dec 2018
Continual Lifelong Learning with Neural Networks: A Review G. I. Parisi Ronald Kemker Jose L. Part Christopher Kanan S. Wermter KELM CLL 203 2,896 0 21 Feb 2018
Continual Learning with Deep Generative Replay Hanul Shin Jung Kwon Lee Jaehong Kim Jiwon Kim KELM CLL 80 2,086 0 24 May 2017
CORe50: a New Dataset and Benchmark for Continuous Object Recognition Vincenzo Lomonaco Davide Maltoni 181 494 0 09 May 2017
iCaRL: Incremental Classifier and Representation Learning Sylvestre-Alvise Rebuffi Alexander Kolesnikov G. Sperl Christoph H. Lampert CLL OOD 160 3,781 0 23 Nov 2016
SQuAD: 100,000+ Questions for Machine Comprehension of Text Pranav Rajpurkar Jian Zhang Konstantin Lopyrev Percy Liang RALM 316 8,169 0 16 Jun 2016