Title
Vibe-Eval: A hard evaluation suite for measuring progress of multimodal language models Piotr Padlewski Max Bain Matthew Henderson Zhongkai Zhu Nishant Relan ... Che Zheng Cyprien de Masson dÁutume Dani Yogatama Mikel Artetxe Yi Tay VLM 89 26 0 03 May 2024
Fewer Truncations Improve Language Modeling Hantian Ding Zijian Wang Giovanni Paolini Varun Kumar Anoop Deoras Dan Roth Stefano Soatto 61 13 0 16 Apr 2024
Goal-guided Generative Prompt Injection Attack on Large Language Models Chong Zhang Mingyu Jin Qinkai Yu Chengzhi Liu Haochen Xue Xiaobo Jin AAML SILM 42 12 0 06 Apr 2024
Beyond Accuracy: Evaluating the Reasoning Behavior of Large Language Models -- A Survey Philipp Mondorf Barbara Plank ELM LRM LM&MA 36 37 0 02 Apr 2024
Scaling Behavior of Machine Translation with Large Language Models under Prompt Injection Attacks Zhifan Sun Antonio Valerio Miceli Barone 39 2 0 14 Mar 2024
Bias-Augmented Consistency Training Reduces Biased Reasoning in Chain-of-Thought James Chua Edward Rees Hunar Batra Samuel R. Bowman Julian Michael Ethan Perez Miles Turpin LRM 47 13 0 08 Mar 2024
On the Essence and Prospect: An Investigation of Alignment Approaches for Big Models Xinpeng Wang Shitong Duan Xiaoyuan Yi Jing Yao Shanlin Zhou Zhihua Wei Peng Zhang Dongkuan Xu Maosong Sun Xing Xie OffRL 41 16 0 07 Mar 2024
Negating Negatives: Alignment without Human Positive Samples via Distributional Dispreference Optimization Shitong Duan Xiaoyuan Yi Peng Zhang Tun Lu Xing Xie Ning Gu 40 4 0 06 Mar 2024
Are More LLM Calls All You Need? Towards Scaling Laws of Compound Inference Systems Lingjiao Chen Jared Quincy Davis Boris Hanin Peter Bailis Ion Stoica Matei A. Zaharia James Zou LRM 34 0 0 04 Mar 2024
Into the Unknown: Self-Learning Large Language Models Teddy Ferdinan Jan Kocoñ P. Kazienko 33 2 0 14 Feb 2024
Suppressing Pink Elephants with Direct Principle Feedback Louis Castricato Nathan Lile Suraj Anand Hailey Schoelkopf Siddharth Verma Stella Biderman 66 10 0 12 Feb 2024
A Tale of Tails: Model Collapse as a Change of Scaling Laws Elvis Dohmatob Yunzhen Feng Pu Yang Francois Charton Julia Kempe 29 66 0 10 Feb 2024
On the generalization capacity of neural networks during generic multimodal reasoning Takuya Ito Soham Dan Mattia Rigotti James Kozloski Murray Campbell LRM 40 2 0 26 Jan 2024
Critical Data Size of Language Models from a Grokking Perspective Xuekai Zhu Yao Fu Bowen Zhou Zhouhan Lin 22 14 0 19 Jan 2024
CHAMP: A Competition-level Dataset for Fine-Grained Analyses of LLMs' Mathematical Reasoning Capabilities Yujun Mao Yoon Kim Yilun Zhou LRM ReLM 26 18 0 13 Jan 2024
Astraios: Parameter-Efficient Instruction Tuning Code Large Language Models Terry Yue Zhuo A. Zebaze Nitchakarn Suppattarachai Leandro von Werra H. D. Vries Qian Liu Niklas Muennighoff ALM 41 15 0 01 Jan 2024
Paloma: A Benchmark for Evaluating Language Model Fit Ian H. Magnusson Akshita Bhagia Valentin Hofmann Luca Soldaini A. Jha ... Iz Beltagy Hanna Hajishirzi Noah A. Smith Kyle Richardson Jesse Dodge 134 21 0 16 Dec 2023
Weak-to-Strong Generalization: Eliciting Strong Capabilities With Weak Supervision Collin Burns Pavel Izmailov Jan Hendrik Kirchner Bowen Baker Leo Gao ... Adrien Ecoffet Manas Joglekar Jan Leike Ilya Sutskever Jeff Wu ELM 50 262 0 14 Dec 2023
Tell, don't show: Declarative facts influence how LLMs generalize Alexander Meinke Owain Evans 26 7 0 12 Dec 2023
A Pipeline For Discourse Circuits From CCG Jonathon Liu R. A. Shaikh Benjamin Rodatz Richie Yeung Bob Coecke 24 2 0 29 Nov 2023
Value FULCRA: Mapping Large Language Models to the Multidimensional Spectrum of Basic Human Values Jing Yao Xiaoyuan Yi Xiting Wang Yifan Gong Xing Xie 41 22 0 15 Nov 2023
In-context Learning Generalizes, But Not Always Robustly: The Case of Syntax Aaron Mueller Albert Webson Jackson Petty Tal Linzen ReLM LRM 36 13 0 13 Nov 2023
Generalization Analogies: A Testbed for Generalizing AI Oversight to Hard-To-Measure Domains Joshua Clymer Garrett Baker Rohan Subramani Sam Wang 22 6 0 13 Nov 2023
Do LLMs exhibit human-like response biases? A case study in survey design Lindia Tjuatja Valerie Chen Sherry Tongshuang Wu Ameet Talwalkar Graham Neubig 35 80 0 07 Nov 2023
A Systematic Comparison of Syllogistic Reasoning in Humans and Language Models Tiwalayo Eisape MH Tessler Ishita Dasgupta Fei Sha Sjoerd van Steenkiste Tal Linzen ReLM LRM 43 8 0 01 Nov 2023
MoCa: Measuring Human-Language Model Alignment on Causal and Moral Judgment Tasks Allen Nie Yuhui Zhang Atharva Amdekar Chris Piech Tatsunori Hashimoto Tobias Gerstenberg 38 36 0 30 Oct 2023
Managing extreme AI risks amid rapid progress Yoshua Bengio Geoffrey Hinton Andrew Yao Dawn Song Pieter Abbeel ... Philip Torr Stuart J. Russell Daniel Kahneman J. Brauner Sören Mindermann 32 63 0 26 Oct 2023
Unpacking the Ethical Value Alignment in Big Models Xiaoyuan Yi Jing Yao Xiting Wang Xing Xie 24 11 0 26 Oct 2023
SoK: Memorization in General-Purpose Large Language Models Valentin Hartmann Anshuman Suri Vincent Bindschaedler David Evans Shruti Tople Robert West KELM LLMAG 26 20 0 24 Oct 2023
ChapGTP, ILLC's Attempt at Raising a BabyLM: Improving Data Efficiency by Automatic Task Formation Jaap Jumelet Michael Hanna Marianne de Heer Kloots Anna Langedijk Charlotte Pouw Oskar van der Wal 29 3 0 17 Oct 2023
Denevil: Towards Deciphering and Navigating the Ethical Values of Large Language Models via Instruction Learning Shitong Duan Xiaoyuan Yi Peng Zhang Tun Lu Xing Xie Ning Gu 24 9 0 17 Oct 2023
In-context Pretraining: Language Modeling Beyond Document Boundaries Weijia Shi Sewon Min Maria Lomeli Chunting Zhou Margaret Li ... Victoria Lin Noah A. Smith Luke Zettlemoyer Scott Yih Mike Lewis LRM RALM SyDa 34 48 0 16 Oct 2023
When can transformers reason with abstract symbols? Enric Boix-Adserà Omid Saremi Emmanuel Abbe Samy Bengio Etai Littwin Josh Susskind LRM NAI 34 12 0 15 Oct 2023
The Consensus Game: Language Model Generation via Equilibrium Search Athul Paul Jacob Songlin Yang Gabriele Farina Jacob Andreas 42 20 0 13 Oct 2023
An Investigation of LLMs' Inefficacy in Understanding Converse Relations Chengwen Qi Bowen Li Binyuan Hui Bailin Wang Jinyang Li Jinwang Wu Yuanjun Laili 32 11 0 08 Oct 2023
Language Models as a Service: Overview of a New Paradigm and its Challenges Emanuele La Malfa Aleksandar Petrov Simon Frieder Christoph Weinhuber Ryan Burnell Raza Nazar Anthony Cohn Nigel Shadbolt Michael Wooldridge ALM ELM 35 3 0 28 Sep 2023
Assessing the nature of large language models: A caution against anthropocentrism Ann Speed ELM AI4MH 21 3 0 14 Sep 2023
Quantifying and Attributing the Hallucination of Large Language Models via Association Analysis Li Du Yequan Wang Xingrun Xing Yiqun Ya Xiang Li Xin Jiang Xuezhi Fang HILM 33 13 0 11 Sep 2023
From Instructions to Intrinsic Human Values -- A Survey of Alignment Goals for Big Models Jing Yao Xiaoyuan Yi Xiting Wang Jindong Wang Xing Xie ALM 27 42 0 23 Aug 2023
CLEVA: Chinese Language Models EVAluation Platform Yanyang Li Jianqiao Zhao Duo Zheng Zi-Yuan Hu Zhi Chen ... Yongfeng Huang Shijia Huang Dahua Lin Michael R. Lyu Liwei Wang ALM ELM 41 10 0 09 Aug 2023
Jailbreak in pieces: Compositional Adversarial Attacks on Multi-Modal Language Models Erfan Shayegani Yue Dong Nael B. Abu-Ghazaleh 47 128 0 26 Jul 2023
Mini-Giants: "Small" Language Models and Open Source Win-Win Zhengping Zhou Lezhi Li Xinxi Chen Andy Li SyDa ALM MoE 32 6 0 17 Jul 2023
Measuring Faithfulness in Chain-of-Thought Reasoning Tamera Lanham Anna Chen Ansh Radhakrishnan Benoit Steiner Carson E. Denison ... Zac Hatfield-Dodds Jared Kaplan J. Brauner Sam Bowman Ethan Perez ReLM LRM 25 167 0 17 Jul 2023
Brain in a Vat: On Missing Pieces Towards Artificial General Intelligence in Large Language Models Yuxi Ma Chi Zhang Song-Chun Zhu ELM ALM 40 8 0 07 Jul 2023
Frontier AI Regulation: Managing Emerging Risks to Public Safety Markus Anderljung Joslyn Barnhart Anton Korinek Jade Leung Cullen O'Keefe ... Jonas Schuett Yonadav Shavit Divya Siddarth Robert F. Trager Kevin J. Wolf SILM 44 118 0 06 Jul 2023
Reasoning or Reciting? Exploring the Capabilities and Limitations of Language Models Through Counterfactual Tasks Zhaofeng Wu Linlu Qiu Alexis Ross Ekin Akyürek Boyuan Chen Bailin Wang Najoung Kim Jacob Andreas Yoon Kim LRM ReLM 58 197 0 05 Jul 2023
Hoodwinked: Deception and Cooperation in a Text-Based Game for Language Models Aidan O'Gara 13 37 0 05 Jul 2023
Probing Quantifier Comprehension in Large Language Models: Another Example of Inverse Scaling Akshat Gupta ELM LRM 24 7 0 12 Jun 2023
Emergent inabilities? Inverse scaling over the course of pretraining J. Michaelov Benjamin Bergen LRM ReLM 22 3 0 24 May 2023
Inverse scaling can become U-shaped Jason W. Wei Najoung Kim Yi Tay Quoc V. Le LRM 29 60 0 03 Nov 2022