v1v2 (latest)

EpiK-Eval: Evaluation for Language Models as Epistemic Models

23 October 2023

Gabriele Prato

Jerry Huang

Prasannna Parthasarathi

Shagun Sodhani

Sarath Chandar

ELM

ArXiv (abs)PDF HTML Github (5★)

Papers citing "EpiK-Eval: Evaluation for Language Models as Epistemic Models"

33 / 33 papers shown

Title
Do Robot Snakes Dream like Electric Sheep? Investigating the Effects of Architectural Inductive Biases on Hallucination Jerry Huang Prasanna Parthasarathi Mehdi Rezagholizadeh Boxing Chen Sarath Chandar 152 0 0 22 Oct 2024
Gorilla: Large Language Model Connected with Massive APIs Shishir G. Patil Tianjun Zhang Xin Wang Joseph E. Gonzalez ELM CLL ALM SyDa 86 556 0 24 May 2023
Reasoning with Language Model is Planning with World Model Shibo Hao Yi Gu Haodi Ma Joshua Jiahua Hong Zhen Wang D. Wang Zhiting Hu ReLM LRM LLMAG 141 579 0 24 May 2023
Unlimiformer: Long-Range Transformers with Unlimited Length Input Amanda Bertsch Uri Alon Graham Neubig Matthew R. Gormley RALM 175 128 0 02 May 2023
Computational Language Acquisition with Theory of Mind Andy Liu Hao Zhu Emmy Liu Yonatan Bisk Graham Neubig LLMAG AI4CE 64 18 0 02 Mar 2023
Hyena Hierarchy: Towards Larger Convolutional Language Models Michael Poli Stefano Massaroli Eric Q. Nguyen Daniel Y. Fu Tri Dao S. Baccus Yoshua Bengio Stefano Ermon Christopher Ré VLM 113 306 0 21 Feb 2023
Adapting Pretrained Text-to-Text Models for Long Text Sequences Wenhan Xiong Anchit Gupta Shubham Toshniwal Yashar Mehdad Wen-tau Yih RALM VLM 91 31 0 21 Sep 2022
PlanBench: An Extensible Benchmark for Evaluating Large Language Models on Planning and Reasoning about Change Karthik Valmeekam Matthew Marquez Alberto Olmo S. Sreedharan Subbarao Kambhampati ReLM LRM 95 236 0 21 Jun 2022
A Review on Language Models as Knowledge Bases Badr AlKhamissi Millicent Li Asli Celikyilmaz Mona T. Diab Marjan Ghazvininejad KELM 84 186 0 12 Apr 2022
Quantifying Memorization Across Neural Language Models Nicholas Carlini Daphne Ippolito Matthew Jagielski Katherine Lee Florian Tramèr Chiyuan Zhang PILM 124 628 0 15 Feb 2022
Training Verifiers to Solve Math Word Problems K. Cobbe V. Kosaraju Mohammad Bavarian Mark Chen Heewoo Jun ... Jerry Tworek Jacob Hilton Reiichiro Nakano Christopher Hesse John Schulman ReLM OffRL LRM 317 4,533 0 27 Oct 2021
Can Language Models be Biomedical Knowledge Bases? Mujeen Sung Jinhyuk Lee Sean S. Yi Minji Jeon Sungdong Kim Jaewoo Kang AI4MH 174 107 0 15 Sep 2021
Finetuned Language Models Are Zero-Shot Learners Jason W. Wei Maarten Bosma Vincent Zhao Kelvin Guu Adams Wei Yu Brian Lester Nan Du Andrew M. Dai Quoc V. Le ALM UQCV 211 3,778 0 03 Sep 2021
Post-hoc Interpretability for Neural NLP: A Survey Andreas Madsen Siva Reddy A. Chandar XAI 74 231 0 10 Aug 2021
Time-Aware Language Models as Temporal Knowledge Bases Bhuwan Dhingra Jeremy R. Cole Julian Martin Eisenschlos D. Gillick Jacob Eisenstein William W. Cohen KELM 95 280 0 29 Jun 2021
Implicit Representations of Meaning in Neural Language Models Belinda Z. Li Maxwell Nye Jacob Andreas NAI MILM 62 177 0 01 Jun 2021
What Makes Good In-Context Examples for GPT- $3$ ? Jiachang Liu Dinghan Shen Yizhe Zhang Bill Dolan Lawrence Carin Weizhu Chen AAML RALM 388 1,387 0 17 Jan 2021
Which Linguist Invented the Lightbulb? Presupposition Verification for Question-Answering Najoung Kim Ellie Pavlick Burcu Karagol Ayan Deepak Ramachandran 129 48 0 02 Jan 2021
Extracting Training Data from Large Language Models Nicholas Carlini Florian Tramèr Eric Wallace Matthew Jagielski Ariel Herbert-Voss ... Tom B. Brown Basel Alomair Ulfar Erlingsson Alina Oprea Colin Raffel MLAU SILM 501 1,943 0 14 Dec 2020
Language Models are Open Knowledge Graphs Chenguang Wang Xiao Liu Basel Alomair SSL KELM 68 137 0 22 Oct 2020
COMET-ATOMIC 2020: On Symbolic and Neural Commonsense Knowledge Graphs Jena D. Hwang Chandra Bhagavatula Ronan Le Bras Jeff Da Keisuke Sakaguchi Antoine Bosselut Yejin Choi 75 411 0 12 Oct 2020
Language Models as Knowledge Bases: On Entity Representations, Storage Capacity, and Paraphrased Queries Benjamin Heinzerling Kentaro Inui KELM 57 133 0 20 Aug 2020
How Much Knowledge Can You Pack Into the Parameters of a Language Model? Adam Roberts Colin Raffel Noam M. Shazeer KELM 121 893 0 10 Feb 2020
REALM: Retrieval-Augmented Language Model Pre-Training Kelvin Guu Kenton Lee Zora Tung Panupong Pasupat Ming-Wei Chang RALM 140 2,114 0 10 Feb 2020
Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer Colin Raffel Noam M. Shazeer Adam Roberts Katherine Lee Sharan Narang Michael Matena Yanqi Zhou Wei Li Peter J. Liu AIMat 450 20,298 0 23 Oct 2019
Language Models as Knowledge Bases? Fabio Petroni Tim Rocktaschel Patrick Lewis A. Bakhtin Yuxiang Wu Alexander H. Miller Sebastian Riedel KELM AI4MH 576 2,673 0 03 Sep 2019
What do you learn from context? Probing for sentence structure in contextualized word representations Ian Tenney Patrick Xia Berlin Chen Alex Jinpeng Wang Adam Poliak ... Najoung Kim Benjamin Van Durme Samuel R. Bowman Dipanjan Das Ellie Pavlick 180 865 0 15 May 2019
Assessing BERT's Syntactic Abilities Yoav Goldberg 73 496 0 16 Jan 2019
Transformer-XL: Attentive Language Models Beyond a Fixed-Length Context Zihang Dai Zhilin Yang Yiming Yang J. Carbonell Quoc V. Le Ruslan Salakhutdinov VLM 250 3,730 0 09 Jan 2019
HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answering Zhilin Yang Peng Qi Saizheng Zhang Yoshua Bengio William W. Cohen Ruslan Salakhutdinov Christopher D. Manning RALM 188 2,689 0 25 Sep 2018
Dissecting Contextual Word Embeddings: Architecture and Representation Matthew E. Peters Mark Neumann Luke Zettlemoyer Wen-tau Yih 99 431 0 27 Aug 2018
Constructing Datasets for Multi-hop Reading Comprehension Across Documents Johannes Welbl Pontus Stenetorp Sebastian Riedel SyDa RALM 98 512 0 17 Oct 2017
A Logic of Knowing How Yanjing Wang 38 52 0 25 May 2015