How Predictable Are Large Language Model Capabilities? A Case Study on
BIG-bench

v1v2 (latest)

How Predictable Are Large Language Model Capabilities? A Case Study on BIG-bench

24 May 2023

Harvey Yiyun Fu

Xiang Ren

Robin Jia

ArXiv (abs)PDF HTML

Papers citing "How Predictable Are Large Language Model Capabilities? A Case Study on BIG-bench"

12 / 12 papers shown

Title
Accelerating Unbiased LLM Evaluation via Synthetic Feedback Zhaoyi Zhou Yuda Song Andrea Zanette ALM 144 0 0 14 Feb 2025
U-shaped and Inverted-U Scaling behind Emergent Abilities of Large Language Models Tung-Yu Wu Pei-Yu Lo ReLM LRM 111 2 0 02 Oct 2024
Challenging BIG-Bench Tasks and Whether Chain-of-Thought Can Solve Them Mirac Suzgun Nathan Scales Nathanael Scharli Sebastian Gehrmann Yi Tay ... Aakanksha Chowdhery Quoc V. Le Ed H. Chi Denny Zhou Jason W. Wei ALM ELM LRM ReLM 266 1,134 0 17 Oct 2022
PaLM: Scaling Language Modeling with Pathways Aakanksha Chowdhery Sharan Narang Jacob Devlin Maarten Bosma Gaurav Mishra ... Kathy Meier-Hellstern Douglas Eck J. Dean Slav Petrov Noah Fiedel PILM LRM 524 6,293 0 05 Apr 2022
Training Compute-Optimal Large Language Models Jordan Hoffmann Sebastian Borgeaud A. Mensch Elena Buchatskaya Trevor Cai ... Karen Simonyan Erich Elsen Jack W. Rae Oriol Vinyals Laurent Sifre AI4TS 208 1,980 0 29 Mar 2022
Do Question Answering Modeling Improvements Hold Across Benchmarks? Nelson F. Liu Tony Lee Robin Jia Percy Liang 55 13 0 01 Feb 2021
Scaling Laws for Neural Language Models Jared Kaplan Sam McCandlish T. Henighan Tom B. Brown B. Chess R. Child Scott Gray Alec Radford Jeff Wu Dario Amodei 611 4,905 0 23 Jan 2020
SuperGLUE: A Stickier Benchmark for General-Purpose Language Understanding Systems Alex Jinpeng Wang Yada Pruksachatkun Nikita Nangia Amanpreet Singh Julian Michael Felix Hill Omer Levy Samuel R. Bowman ELM 274 2,323 0 02 May 2019
GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding Alex Jinpeng Wang Amanpreet Singh Julian Michael Felix Hill Omer Levy Samuel R. Bowman ELM 1.1K 7,196 0 20 Apr 2018
SentEval: An Evaluation Toolkit for Universal Sentence Representations Alexis Conneau Douwe Kiela 100 641 0 14 Mar 2018
SQuAD: 100,000+ Questions for Machine Comprehension of Text Pranav Rajpurkar Jian Zhang Konstantin Lopyrev Percy Liang RALM 316 8,169 0 16 Jun 2016
A large annotated corpus for learning natural language inference Samuel R. Bowman Gabor Angeli Christopher Potts Christopher D. Manning 324 4,294 0 21 Aug 2015

We use cookies and other tracking technologies to improve your browsing experience on our website, to show you personalized content and targeted ads, to analyze our website traffic, and to understand where our visitors are coming from. See our policy.