v1v2 (latest)

Towards Better Few-Shot and Finetuning Performance with Forgetful Causal Language Models

24 October 2022

Sharan Narang

Pieter Abbeel

KELM

CLL

ArXiv (abs)PDF HTML

Papers citing "Towards Better Few-Shot and Finetuning Performance with Forgetful Causal Language Models"

31 / 31 papers shown

Title
Efficient Training of Language Models to Fill in the Middle Mohammad Bavarian Heewoo Jun Nikolas Tezak John Schulman C. McLeavey Jerry Tworek Mark Chen 89 197 0 28 Jul 2022
Solving Quantitative Reasoning Problems with Language Models Aitor Lewkowycz Anders Andreassen David Dohan Ethan Dyer Henryk Michalewski ... Theo Gutman-Solo Yuhuai Wu Behnam Neyshabur Guy Gur-Ari Vedant Misra ReLM ELM LRM 181 859 0 29 Jun 2022
Flamingo: a Visual Language Model for Few-Shot Learning Jean-Baptiste Alayrac Jeff Donahue Pauline Luc Antoine Miech Iain Barr ... Mikolaj Binkowski Ricardo Barreira Oriol Vinyals Andrew Zisserman Karen Simonyan MLLM VLM 418 3,610 0 29 Apr 2022
GPT-NeoX-20B: An Open-Source Autoregressive Language Model Sid Black Stella Biderman Eric Hallahan Quentin G. Anthony Leo Gao ... Shivanshu Purohit Laria Reynolds J. Tow Benqi Wang Samuel Weinbach 183 836 0 14 Apr 2022
What Language Model Architecture and Pretraining Objective Work Best for Zero-Shot Generalization? Thomas Wang Adam Roberts Daniel Hesslow Teven Le Scao Hyung Won Chung Iz Beltagy Julien Launay Colin Raffel 129 176 0 12 Apr 2022
PaLM: Scaling Language Modeling with Pathways Aakanksha Chowdhery Sharan Narang Jacob Devlin Maarten Bosma Gaurav Mishra ... Kathy Meier-Hellstern Douglas Eck J. Dean Slav Petrov Noah Fiedel PILM LRM 535 6,301 0 05 Apr 2022
Do As I Can, Not As I Say: Grounding Language in Robotic Affordances Michael Ahn Anthony Brohan Noah Brown Yevgen Chebotar Omar Cortes ... Ted Xiao Peng Xu Sichun Xu Mengyuan Yan Andy Zeng LM&Ro 195 1,988 0 04 Apr 2022
$Scaling Up Models and Data with $\texttt{t5x}$ and $\texttt{seqio}$$ Scaling Up Models and Data with $\texttt{t5x}$ and $\texttt{seqio}$ Adam Roberts Hyung Won Chung Anselm Levskaya Gaurav Mishra James Bradbury ... Brennan Saeta Ryan Sepassi A. Spiridonov Joshua Newlan Andrea Gesmundo ALM 126 199 0 31 Mar 2022
Training Compute-Optimal Large Language Models Jordan Hoffmann Sebastian Borgeaud A. Mensch Elena Buchatskaya Trevor Cai ... Karen Simonyan Erich Elsen Jack W. Rae Oriol Vinyals Laurent Sifre AI4TS 208 1,987 0 29 Mar 2022
Multitask Prompted Training Enables Zero-Shot Task Generalization Victor Sanh Albert Webson Colin Raffel Stephen H. Bach Lintang Sutawika ... T. Bers Stella Biderman Leo Gao Thomas Wolf Alexander M. Rush LRM 355 1,710 0 15 Oct 2021
Pre-train, Prompt, and Predict: A Systematic Survey of Prompting Methods in Natural Language Processing Pengfei Liu Weizhe Yuan Jinlan Fu Zhengbao Jiang Hiroaki Hayashi Graham Neubig VLM SyDa 239 4,004 0 28 Jul 2021
Evaluating Large Language Models Trained on Code Mark Chen Jerry Tworek Heewoo Jun Qiming Yuan Henrique Pondé ... Bob McGrew Dario Amodei Sam McCandlish Ilya Sutskever Wojciech Zaremba ELM ALM 238 5,675 0 07 Jul 2021
RoFormer: Enhanced Transformer with Rotary Position Embedding Jianlin Su Yu Lu Shengfeng Pan Ahmed Murtadha Bo Wen Yunfeng Liu 329 2,533 0 20 Apr 2021
Cross-Task Generalization via Natural Language Crowdsourcing Instructions Swaroop Mishra Daniel Khashabi Chitta Baral Hannaneh Hajishirzi LRM 173 753 0 18 Apr 2021
The Power of Scale for Parameter-Efficient Prompt Tuning Brian Lester Rami Al-Rfou Noah Constant VPVLM 600 4,099 0 18 Apr 2021
GLM: General Language Model Pretraining with Autoregressive Blank Infilling Zhengxiao Du Yujie Qian Xiao Liu Ming Ding J. Qiu Zhilin Yang Jie Tang BDL AI4CE 151 1,556 0 18 Mar 2021
Prefix-Tuning: Optimizing Continuous Prompts for Generation Xiang Lisa Li Percy Liang 252 4,305 0 01 Jan 2021
Enabling Language Models to Fill in the Blanks Chris Donahue Mina Lee Percy Liang 51 198 0 11 May 2020
SuperGlue: Learning Feature Matching with Graph Neural Networks Paul-Edouard Sarlin Daniel DeTone Tomasz Malisiewicz Andrew Rabinovich 3DPC OffRL 143 1,950 0 26 Nov 2019
PIQA: Reasoning about Physical Commonsense in Natural Language Yonatan Bisk Rowan Zellers Ronan Le Bras Jianfeng Gao Yejin Choi OOD LRM 186 1,847 0 26 Nov 2019
Quick and (not so) Dirty: Unsupervised Selection of Justification Sentences for Multi-hop Question Answering Vikas Yadav Steven Bethard Mihai Surdeanu 127 77 0 17 Nov 2019
Fast Transformer Decoding: One Write-Head is All You Need Noam M. Shazeer 161 476 0 06 Nov 2019
Adversarial NLI: A New Benchmark for Natural Language Understanding Yixin Nie Adina Williams Emily Dinan Joey Tianyi Zhou Jason Weston Douwe Kiela 127 1,011 0 31 Oct 2019
XLNet: Generalized Autoregressive Pretraining for Language Understanding Zhilin Yang Zihang Dai Yiming Yang J. Carbonell Ruslan Salakhutdinov Quoc V. Le AI4CE 236 8,455 0 19 Jun 2019
Unified Language Model Pre-training for Natural Language Understanding and Generation Li Dong Nan Yang Wenhui Wang Furu Wei Xiaodong Liu Yu Wang Jianfeng Gao M. Zhou H. Hon ELM AI4CE 230 1,560 0 08 May 2019
SentencePiece: A simple and language independent subword tokenizer and detokenizer for Neural Text Processing Taku Kudo John Richardson 209 3,534 0 19 Aug 2018
Adafactor: Adaptive Learning Rates with Sublinear Memory Cost Noam M. Shazeer Mitchell Stern ODL 86 1,053 0 11 Apr 2018
Deep contextualized word representations Matthew E. Peters Mark Neumann Mohit Iyyer Matt Gardner Christopher Clark Kenton Lee Luke Zettlemoyer NAI 233 11,569 0 15 Feb 2018
Program Induction by Rationale Generation : Learning to Solve and Explain Algebraic Word Problems Wang Ling Dani Yogatama Chris Dyer Phil Blunsom AIMat 109 737 0 11 May 2017
Generating Sentences from a Continuous Space Samuel R. Bowman Luke Vilnis Oriol Vinyals Andrew M. Dai Rafal Jozefowicz Samy Bengio DRL 113 2,365 0 19 Nov 2015
Scheduled Sampling for Sequence Prediction with Recurrent Neural Networks Samy Bengio Oriol Vinyals Navdeep Jaitly Noam M. Shazeer 156 2,039 0 09 Jun 2015