Navigating the Landscape of Large Language Models: A Comprehensive Review and Analysis of Paradigms and Fine-Tuning Strategies

13 April 2024

Papers citing "Navigating the Landscape of Large Language Models: A Comprehensive Review and Analysis of Paradigms and Fine-Tuning Strategies"

50 / 63 papers shown

Title
Nash Learning from Human Feedback Rémi Munos Michal Valko Daniele Calandriello M. G. Azar Mark Rowland ... Nikola Momchev Olivier Bachem D. Mankowitz Doina Precup Bilal Piot 102 147 0 01 Dec 2023
Igniting Language Intelligence: The Hitchhiker's Guide From Chain-of-Thought Reasoning to Language Agents Zhuosheng Zhang Yao Yao Aston Zhang Xiangru Tang Xinbei Ma ... Yiming Wang Mark B. Gerstein Rui Wang Gongshen Liu Hai Zhao LLMAG LM&Ro LRM 144 60 0 20 Nov 2023
Prompt Engineering a Prompt Engineer Qinyuan Ye Maxamed Axmed Reid Pryzant Fereshte Khani VLM LLMAG LRM 85 32 0 09 Nov 2023
AgentTuning: Enabling Generalized Agent Abilities for LLMs Aohan Zeng Mingdao Liu Rui Lu Bowen Wang Xiao Liu Yuxiao Dong Jie Tang LM&MA ALM LLMAG 105 185 0 19 Oct 2023
Direct Preference Optimization: Your Language Model is Secretly a Reward Model Rafael Rafailov Archit Sharma E. Mitchell Stefano Ermon Christopher D. Manning Chelsea Finn ALM 389 4,163 0 29 May 2023
InstructUIE: Multi-task Instruction Tuning for Unified Information Extraction Xiao Wang Wei Zhou Can Zu Han Xia Tianze Chen ... Tao Gui Jihua Kang J. Yang Siyuan Li Chunsai Du 75 158 0 17 Apr 2023
Scaling Down to Scale Up: A Guide to Parameter-Efficient Fine-Tuning Vladislav Lialin Vijeta Deshpande Anna Rumshisky 100 177 0 28 Mar 2023
Adding Conditional Control to Text-to-Image Diffusion Models Lvmin Zhang Anyi Rao Maneesh Agrawala AI4CE 184 4,175 1 10 Feb 2023
REPLUG: Retrieval-Augmented Black-Box Language Models Weijia Shi Sewon Min Michihiro Yasunaga Minjoon Seo Rich James M. Lewis Luke Zettlemoyer Wen-tau Yih RALM VLM KELM 160 642 0 30 Jan 2023
Memory-efficient NLLB-200: Language-specific Expert Pruning of a Massively Multilingual Machine Translation Model Yeskendir Koishekenov Alexandre Berard Vassilina Nikoulina MoE 68 30 0 19 Dec 2022
Robust Speech Recognition via Large-Scale Weak Supervision Alec Radford Jong Wook Kim Tao Xu Greg Brockman C. McLeavey Ilya Sutskever OffRL 215 3,750 0 06 Dec 2022
Galvatron: Efficient Transformer Training over Multiple GPUs Using Automatic Parallelism Xupeng Miao Yujie Wang Youhe Jiang Chunan Shi Xiaonan Nie Hailin Zhang Tengjiao Wang GNN MoE 98 64 0 25 Nov 2022
BLOOM: A 176B-Parameter Open-Access Multilingual Language Model BigScience Workshop : Teven Le Scao Angela Fan Christopher Akiki ... Zhongli Xie Zifan Ye M. Bras Younes Belkada Thomas Wolf VLM 408 2,393 0 09 Nov 2022
Scaling Instruction-Finetuned Language Models Hyung Won Chung Le Hou Shayne Longpre Barret Zoph Yi Tay ... Jacob Devlin Adam Roberts Denny Zhou Quoc V. Le Jason W. Wei ReLM LRM 231 3,158 0 20 Oct 2022
Improving the Domain Adaptation of Retrieval Augmented Generation (RAG) Models for Open Domain Question Answering Shamane Siriwardhana Rivindu Weerasekera Elliott Wen Tharindu Kaluarachchi R. Rana Suranga Nanayakkara VLM 70 182 0 06 Oct 2022
ReAct: Synergizing Reasoning and Acting in Language Models Shunyu Yao Jeffrey Zhao Dian Yu Nan Du Izhak Shafran Karthik Narasimhan Yuan Cao LLMAG ReLM LRM 439 2,976 0 06 Oct 2022
GLM-130B: An Open Bilingual Pre-trained Model Aohan Zeng Xiao Liu Zhengxiao Du Zihan Wang Hanyu Lai ... Jidong Zhai Wenguang Chen Peng Zhang Yuxiao Dong Jie Tang BDL LRM 362 1,094 0 05 Oct 2022
BEiT v2: Masked Image Modeling with Vector-Quantized Visual Tokenizers Zhiliang Peng Li Dong Hangbo Bao QiXiang Ye Furu Wei 69 320 0 12 Aug 2022
CogView2: Faster and Better Text-to-Image Generation via Hierarchical Transformers Ming Ding Wendi Zheng Wenyi Hong Jie Tang VLM 123 334 0 28 Apr 2022
GPT-NeoX-20B: An Open-Source Autoregressive Language Model Sid Black Stella Biderman Eric Hallahan Quentin G. Anthony Leo Gao ... Shivanshu Purohit Laria Reynolds J. Tow Benqi Wang Samuel Weinbach 183 836 0 14 Apr 2022
Hierarchical Text-Conditional Image Generation with CLIP Latents Aditya A. Ramesh Prafulla Dhariwal Alex Nichol Casey Chu Mark Chen VLM DiffM 413 6,916 0 13 Apr 2022
PaLM: Scaling Language Modeling with Pathways Aakanksha Chowdhery Sharan Narang Jacob Devlin Maarten Bosma Gaurav Mishra ... Kathy Meier-Hellstern Douglas Eck J. Dean Slav Petrov Noah Fiedel PILM LRM 529 6,293 0 05 Apr 2022
CICERO: A Dataset for Contextualized Commonsense Inference in Dialogues Deepanway Ghosal Siqi Shen Navonil Majumder Rada Mihalcea Soujanya Poria 105 53 0 25 Mar 2022
Training language models to follow instructions with human feedback Long Ouyang Jeff Wu Xu Jiang Diogo Almeida Carroll L. Wainwright ... Amanda Askell Peter Welinder Paul Christiano Jan Leike Ryan J. Lowe OSLM ALM 886 13,207 0 04 Mar 2022
Rethinking the Role of Demonstrations: What Makes In-Context Learning Work? Sewon Min Xinxi Lyu Ari Holtzman Mikel Artetxe M. Lewis Hannaneh Hajishirzi Luke Zettlemoyer LLMAG LRM 187 1,498 0 25 Feb 2022
Competition-Level Code Generation with AlphaCode Yujia Li David Choi Junyoung Chung Nate Kushman Julian Schrittwieser ... Esme Sutherland Robson Pushmeet Kohli Nando de Koray Kavukcuoglu Oriol Vinyals 148 1,425 0 08 Feb 2022
Chain-of-Thought Prompting Elicits Reasoning in Large Language Models Jason W. Wei Xuezhi Wang Dale Schuurmans Maarten Bosma Brian Ichter F. Xia Ed H. Chi Quoc Le Denny Zhou LM&Ro LRM AI4CE ReLM 850 9,683 0 28 Jan 2022
LaMDA: Language Models for Dialog Applications R. Thoppilan Daniel De Freitas Jamie Hall Noam M. Shazeer Apoorv Kulshreshtha ... Blaise Aguera-Arcas Claire Cui M. Croak Ed H. Chi Quoc Le ALM 144 1,601 0 20 Jan 2022
High-Resolution Image Synthesis with Latent Diffusion Models Robin Rombach A. Blattmann Dominik Lorenz Patrick Esser Bjorn Ommer 3DV 502 15,788 0 20 Dec 2021
GLaM: Efficient Scaling of Language Models with Mixture-of-Experts Nan Du Yanping Huang Andrew M. Dai Simon Tong Dmitry Lepikhin ... Kun Zhang Quoc V. Le Yonghui Wu Zhiwen Chen Claire Cui ALM MoE 231 826 0 13 Dec 2021
An Explanation of In-context Learning as Implicit Bayesian Inference Sang Michael Xie Aditi Raghunathan Percy Liang Tengyu Ma ReLM BDL VPVLM LRM 225 764 0 03 Nov 2021
UniPELT: A Unified Framework for Parameter-Efficient Language Model Tuning Yuning Mao Lambert Mathias Rui Hou Amjad Almahairi Hao Ma Jiawei Han Wen-tau Yih Madian Khabsa 66 193 0 14 Oct 2021
Yuan 1.0: Large-Scale Pre-trained Language Model in Zero-Shot and Few-Shot Learning Shaohua Wu Xudong Zhao Tong Yu Rongguo Zhang C. Shen ... Feng Li Hong Zhu Jiangang Luo Liang Xu Xuanwei Zhang ALM 51 60 0 10 Oct 2021
Towards a Unified View of Parameter-Efficient Transfer Learning Junxian He Chunting Zhou Xuezhe Ma Taylor Berg-Kirkpatrick Graham Neubig AAML 137 953 0 08 Oct 2021
Finetuned Language Models Are Zero-Shot Learners Jason W. Wei Maarten Bosma Vincent Zhao Kelvin Guu Adams Wei Yu Brian Lester Nan Du Andrew M. Dai Quoc V. Le ALM UQCV 246 3,789 0 03 Sep 2021
Evaluating Large Language Models Trained on Code Mark Chen Jerry Tworek Heewoo Jun Qiming Yuan Henrique Pondé ... Bob McGrew Dario Amodei Sam McCandlish Ilya Sutskever Wojciech Zaremba ELM ALM 236 5,665 0 07 Jul 2021
ERNIE 3.0: Large-scale Knowledge Enhanced Pre-training for Language Understanding and Generation Yu Sun Shuohuan Wang Shikun Feng Siyu Ding Chao Pang ... Ouyang Xuan Dianhai Yu Hao Tian Hua Wu Haifeng Wang 112 472 0 05 Jul 2021
CPM-2: Large-scale Cost-effective Pre-trained Language Models Zhengyan Zhang Yuxian Gu Xu Han Shengqi Chen Chaojun Xiao ... Minlie Huang Wentao Han Yang Liu Xiaoyan Zhu Maosong Sun MoE 78 87 0 20 Jun 2021
BitFit: Simple Parameter-efficient Fine-tuning for Transformer-based Masked Language-models Elad Ben-Zaken Shauli Ravfogel Yoav Goldberg 187 1,243 0 18 Jun 2021
LoRA: Low-Rank Adaptation of Large Language Models J. E. Hu Yelong Shen Phillip Wallis Zeyuan Allen-Zhu Yuanzhi Li Shean Wang Lu Wang Weizhu Chen OffRL AI4TS AI4CE ALM AIMat 502 10,526 0 17 Jun 2021
BEiT: BERT Pre-Training of Image Transformers Hangbo Bao Li Dong Songhao Piao Furu Wei ViT 292 2,845 0 15 Jun 2021
CogView: Mastering Text-to-Image Generation via Transformers Ming Ding Zhuoyi Yang Wenyi Hong Wendi Zheng Chang Zhou ... Junyang Lin Xu Zou Zhou Shao Hongxia Yang Jie Tang ViT VLM 125 782 0 26 May 2021
RoFormer: Enhanced Transformer with Rotary Position Embedding Jianlin Su Yu Lu Shengfeng Pan Ahmed Murtadha Bo Wen Yunfeng Liu 327 2,533 0 20 Apr 2021
The Power of Scale for Parameter-Efficient Prompt Tuning Brian Lester Rami Al-Rfou Noah Constant VPVLM 589 4,093 0 18 Apr 2021
GPT Understands, Too Xiao Liu Yanan Zheng Zhengxiao Du Ming Ding Yujie Qian Zhilin Yang Jie Tang VLM 168 1,182 0 18 Mar 2021
GLM: General Language Model Pretraining with Autoregressive Blank Infilling Zhengxiao Du Yujie Qian Xiao Liu Ming Ding J. Qiu Zhilin Yang Jie Tang BDL AI4CE 151 1,554 0 18 Mar 2021
Prefix-Tuning: Optimizing Continuous Prompts for Generation Xiang Lisa Li Percy Liang 252 4,305 0 01 Jan 2021
Continual Learning for Natural Language Generation in Task-oriented Dialog Systems Fei Mi Liangwei Chen Mengjie Zhao Minlie Huang Boi Faltings CLL KELM 50 71 0 02 Oct 2020
GShard: Scaling Giant Models with Conditional Computation and Automatic Sharding Dmitry Lepikhin HyoukJoong Lee Yuanzhong Xu Dehao Chen Orhan Firat Yanping Huang M. Krikun Noam M. Shazeer Zhiwen Chen MoE 127 1,191 0 30 Jun 2020
Fast Transformer Decoding: One Write-Head is All You Need Noam M. Shazeer 161 476 0 06 Nov 2019