A Latent Space Theory for Emergent Abilities in Large Language Models

v1v2v3 (latest)

A Latent Space Theory for Emergent Abilities in Large Language Models

19 April 2023

ArXiv (abs)PDF HTML

Papers citing "A Latent Space Theory for Emergent Abilities in Large Language Models"

15 / 15 papers shown

Title
Fast Gradient Computation for RoPE Attention in Almost Linear Time Yifang Chen Jiayan Huo Xiaoyu Li Yingyu Liang Zhenmei Shi Zhao Song 122 14 0 03 Jan 2025
Demystifying Chains, Trees, and Graphs of Thoughts Maciej Besta Florim Memedi Zhenyu Zhang Robert Gerstenberger Guangyuan Piao ... Aleš Kubíček H. Niewiadomski Aidan O'Mahony Onur Mutlu Torsten Hoefler AI4CE LRM 254 33 0 25 Jan 2024
A Theory of Emergent In-Context Learning as Implicit Structure Induction Michael Hahn Navin Goyal LRM 56 85 0 14 Mar 2023
Scaling Instruction-Finetuned Language Models Hyung Won Chung Le Hou Shayne Longpre Barret Zoph Yi Tay ... Jacob Devlin Adam Roberts Denny Zhou Quoc V. Le Jason W. Wei ReLM LRM 194 3,146 0 20 Oct 2022
Emergent Abilities of Large Language Models Jason W. Wei Yi Tay Rishi Bommasani Colin Raffel Barret Zoph ... Tatsunori Hashimoto Oriol Vinyals Percy Liang J. Dean W. Fedus ELM ReLM LRM 286 2,507 0 15 Jun 2022
Data Distributional Properties Drive Emergent In-Context Learning in Transformers Stephanie C. Y. Chan Adam Santoro Andrew Kyle Lampinen Jane X. Wang Aaditya K. Singh Pierre Harvey Richemond J. Mcclelland Felix Hill 145 265 0 22 Apr 2022
PaLM: Scaling Language Modeling with Pathways Aakanksha Chowdhery Sharan Narang Jacob Devlin Maarten Bosma Gaurav Mishra ... Kathy Meier-Hellstern Douglas Eck J. Dean Slav Petrov Noah Fiedel PILM LRM 509 6,279 0 05 Apr 2022
Impact of Pretraining Term Frequencies on Few-Shot Reasoning Yasaman Razeghi Robert L Logan IV Matt Gardner Sameer Singh ReLM LRM 86 156 0 15 Feb 2022
Learning To Retrieve Prompts for In-Context Learning Ohad Rubin Jonathan Herzig Jonathan Berant VPVLM RALM 86 707 0 16 Dec 2021
An Explanation of In-context Learning as Implicit Bayesian Inference Sang Michael Xie Aditi Raghunathan Percy Liang Tengyu Ma ReLM BDL VPVLM LRM 204 759 0 03 Nov 2021
Multitask Prompted Training Enables Zero-Shot Task Generalization Victor Sanh Albert Webson Colin Raffel Stephen H. Bach Lintang Sutawika ... T. Bers Stella Biderman Leo Gao Thomas Wolf Alexander M. Rush LRM 346 1,706 0 15 Oct 2021
Finetuned Language Models Are Zero-Shot Learners Jason W. Wei Maarten Bosma Vincent Zhao Kelvin Guu Adams Wei Yu Brian Lester Nan Du Andrew M. Dai Quoc V. Le ALM UQCV 211 3,778 0 03 Sep 2021
What Makes Good In-Context Examples for GPT- $3$ ? Jiachang Liu Dinghan Shen Yizhe Zhang Bill Dolan Lawrence Carin Weizhu Chen AAML RALM 385 1,387 0 17 Jan 2021
On Approximation Capabilities of ReLU Activation and Softmax Output Layer in Neural Networks Behnam Asadi Hui Jiang 44 20 0 10 Feb 2020
Are Transformers universal approximators of sequence-to-sequence functions? Chulhee Yun Srinadh Bhojanapalli A. S. Rawat Sashank J. Reddi Sanjiv Kumar 115 357 0 20 Dec 2019