v1v2 (latest)

LMentry: A Language Model Benchmark of Elementary Language Tasks

3 November 2022

Papers citing "LMentry: A Language Model Benchmark of Elementary Language Tasks"

29 / 29 papers shown

Title
On the Relation between Sensitivity and Accuracy in In-context Learning Yanda Chen Chen Zhao Zhou Yu Kathleen McKeown He He 244 80 0 16 Sep 2022
Beyond the Imitation Game: Quantifying and extrapolating the capabilities of language models Aarohi Srivastava Abhinav Rastogi Abhishek Rao Abu Awal Md Shoeb Abubakar Abid ... Zhuoye Zhao Zijian Wang Zijie J. Wang Zirui Wang Ziyi Wu ELM 211 1,775 0 09 Jun 2022
OPT: Open Pre-trained Transformer Language Models Susan Zhang Stephen Roller Naman Goyal Mikel Artetxe Moya Chen ... Daniel Simig Punit Singh Koura Anjali Sridhar Tianlu Wang Luke Zettlemoyer VLM OSLM AI4CE 362 3,699 0 02 May 2022
PaLM: Scaling Language Modeling with Pathways Aakanksha Chowdhery Sharan Narang Jacob Devlin Maarten Bosma Gaurav Mishra ... Kathy Meier-Hellstern Douglas Eck J. Dean Slav Petrov Noah Fiedel PILM LRM 529 6,293 0 05 Apr 2022
Training language models to follow instructions with human feedback Long Ouyang Jeff Wu Xu Jiang Diogo Almeida Carroll L. Wainwright ... Amanda Askell Peter Welinder Paul Christiano Jan Leike Ryan J. Lowe OSLM ALM 886 13,207 0 04 Mar 2022
Rethinking the Role of Demonstrations: What Makes In-Context Learning Work? Sewon Min Xinxi Lyu Ari Holtzman Mikel Artetxe M. Lewis Hannaneh Hajishirzi Luke Zettlemoyer LLMAG LRM 172 1,498 0 25 Feb 2022
PromptSource: An Integrated Development Environment and Repository for Natural Language Prompts Stephen H. Bach Victor Sanh Zheng-Xin Yong Albert Webson Colin Raffel ... Khalid Almubarak Xiangru Tang Dragomir R. Radev Mike Tian-Jian Jiang Alexander M. Rush VLM 333 349 0 02 Feb 2022
Using DeepSpeed and Megatron to Train Megatron-Turing NLG 530B, A Large-Scale Generative Language Model Shaden Smith M. Patwary Brandon Norick P. LeGresley Samyam Rajbhandari ... Mohammad Shoeybi Yuxiong He Michael Houston Saurabh Tiwary Bryan Catanzaro MoE 155 742 0 28 Jan 2022
Scaling Language Models: Methods, Analysis & Insights from Training Gopher Jack W. Rae Sebastian Borgeaud Trevor Cai Katie Millican Jordan Hoffmann ... Jeff Stanway L. Bennett Demis Hassabis Koray Kavukcuoglu G. Irving 136 1,325 0 08 Dec 2021
Multitask Prompted Training Enables Zero-Shot Task Generalization Victor Sanh Albert Webson Colin Raffel Stephen H. Bach Lintang Sutawika ... T. Bers Stella Biderman Leo Gao Thomas Wolf Alexander M. Rush LRM 355 1,709 0 15 Oct 2021
Finetuned Language Models Are Zero-Shot Learners Jason W. Wei Maarten Bosma Vincent Zhao Kelvin Guu Adams Wei Yu Brian Lester Nan Du Andrew M. Dai Quoc V. Le ALM UQCV 235 3,782 0 03 Sep 2021
Language Model Evaluation Beyond Perplexity Clara Meister Ryan Cotterell 133 77 0 31 May 2021
True Few-Shot Learning with Language Models Ethan Perez Douwe Kiela Kyunghyun Cho 137 439 0 24 May 2021
Fantastically Ordered Prompts and Where to Find Them: Overcoming Few-Shot Prompt Order Sensitivity Yao Lu Max Bartolo Alastair Moore Sebastian Riedel Pontus Stenetorp AILaw LRM 412 1,195 0 18 Apr 2021
Cross-Task Generalization via Natural Language Crowdsourcing Instructions Swaroop Mishra Daniel Khashabi Chitta Baral Hannaneh Hajishirzi LRM 171 752 0 18 Apr 2021
Self-Diagnosis and Self-Debiasing: A Proposal for Reducing Corpus-Based Bias in NLP Timo Schick Sahana Udupa Hinrich Schütze 313 387 0 28 Feb 2021
The GEM Benchmark: Natural Language Generation, its Evaluation and Metrics Sebastian Gehrmann Tosin Adewumi Karmanya Aggarwal Pawan Sasanka Ammanamanchi Aremu Anuoluwapo ... Nishant Subramani Wei Xu Diyi Yang Akhila Yerukola Jiawei Zhou VLM 311 285 0 02 Feb 2021
Measuring Massive Multitask Language Understanding Dan Hendrycks Collin Burns Steven Basart Andy Zou Mantas Mazeika Basel Alomair Jacob Steinhardt ELM RALM 184 4,572 0 07 Sep 2020
Learning to summarize from human feedback Nisan Stiennon Long Ouyang Jeff Wu Daniel M. Ziegler Ryan J. Lowe Chelsea Voss Alec Radford Dario Amodei Paul Christiano ALM 259 2,189 0 02 Sep 2020
Language Models are Few-Shot Learners Tom B. Brown Benjamin Mann Nick Ryder Melanie Subbiah Jared Kaplan ... Christopher Berner Sam McCandlish Alec Radford Ilya Sutskever Dario Amodei BDL 882 42,463 0 28 May 2020
Beyond Accuracy: Behavioral Testing of NLP models with CheckList Marco Tulio Ribeiro Tongshuang Wu Carlos Guestrin Sameer Singh ELM 210 1,110 0 08 May 2020
oLMpics -- On what Language Model Pre-training Captures Alon Talmor Yanai Elazar Yoav Goldberg Jonathan Berant LRM 103 304 0 31 Dec 2019
Reducing Sentiment Bias in Language Models via Counterfactual Evaluation Po-Sen Huang Huan Zhang Ray Jiang Robert Stanforth Johannes Welbl Jack W. Rae Vishal Maini Dani Yogatama Pushmeet Kohli 89 217 0 08 Nov 2019
Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer Colin Raffel Noam M. Shazeer Adam Roberts Katherine Lee Sharan Narang Michael Matena Yanqi Zhou Wei Li Peter J. Liu AIMat 488 20,342 0 23 Oct 2019
Identifying and Reducing Gender Bias in Word-Level Language Models Shikha Bordia Samuel R. Bowman FaML 120 327 0 05 Apr 2019
Right for the Wrong Reasons: Diagnosing Syntactic Heuristics in Natural Language Inference R. Thomas McCoy Ellie Pavlick Tal Linzen 143 1,244 0 04 Feb 2019
Neural Network Acceptability Judgments Alex Warstadt Amanpreet Singh Samuel R. Bowman 242 1,413 0 31 May 2018
Deep reinforcement learning from human preferences Paul Christiano Jan Leike Tom B. Brown Miljan Martic Shane Legg Dario Amodei 218 3,377 0 12 Jun 2017
Towards AI-Complete Question Answering: A Set of Prerequisite Toy Tasks Jason Weston Antoine Bordes S. Chopra Alexander M. Rush Bart van Merriënboer Armand Joulin Tomas Mikolov LRM ELM 153 1,182 0 19 Feb 2015