How Can We Know What Language Models Know?

28 November 2019

Graham Neubig

Papers citing "How Can We Know What Language Models Know?"

50 / 301 papers shown

Title
RAPTOR: Recursive Abstractive Processing for Tree-Organized Retrieval Parth Sarthi Salman Abdullah Aditi Tuli Shubh Khanna Anna Goldie Christopher D. Manning RALM 24 123 0 31 Jan 2024
Desiderata for the Context Use of Question Answering Systems Sagi Shaier Lawrence E Hunter K. Wense 28 4 0 31 Jan 2024
In-context Learning with Retrieved Demonstrations for Language Models: A Survey an Luo Xin Xu Yue Liu Panupong Pasupat Mehran Kazemi RALM 34 55 0 21 Jan 2024
Prompting open-source and commercial language models for grammatical error correction of English learner text Christopher Davis Andrew Caines Oistein Andersen Shiva Taslimipoor H. Yannakoudakis Zheng Yuan Christopher Bryant Marek Rei P. Buttery 35 13 0 15 Jan 2024
How Proficient Are Large Language Models in Formal Languages? An In-Depth Insight for Knowledge Base Question Answering Jinxi Liu S. Cao Jiaxin Shi Tingjian Zhang Lunyiu Nie Linmei Hu Lei Hou Juanzi Li ELM 33 3 0 11 Jan 2024
The Butterfly Effect of Altering Prompts: How Small Changes and Jailbreaks Affect Large Language Model Performance A. Salinas Fred Morstatter 45 50 0 08 Jan 2024
CLadder: Assessing Causal Reasoning in Language Models Zhijing Jin Yuen Chen Felix Leeb Luigi Gresele Ojasv Kamal ... Kevin Blin Fernando Gonzalez Adauto Max Kleiman-Weiner Mrinmaya Sachan Bernhard Schölkopf ReLM ELM LRM 51 64 0 07 Dec 2023
Assertion Enhanced Few-Shot Learning: Instructive Technique for Large Language Models to Generate Educational Explanations Tasmia Shahriar Kelly Ramos Noboru Matsuda 32 3 0 05 Dec 2023
Strings from the Library of Babel: Random Sampling as a Strong Baseline for Prompt Optimisation Yao Lu Jiayi Wang Raphael Tang Sebastian Riedel Pontus Stenetorp 38 2 0 16 Nov 2023
Assessing Knowledge Editing in Language Models via Relation Perspective Yifan Wei Xiaoyan Yu Huanhuan Ma Fangyu Lei Yixuan Weng Ran Song Kang Liu KELM 39 15 0 15 Nov 2023
Can Knowledge Graphs Reduce Hallucinations in LLMs? : A Survey Garima Agrawal Tharindu Kumarage Zeyad Alghami Huanmin Liu 37 81 0 14 Nov 2023
Evaluating General-Purpose AI with Psychometrics Xiting Wang Liming Jiang Jose Hernandez-Orallo David Stillwell Luning Sun Fang Luo Xing Xie AI4MH ELM 38 12 0 25 Oct 2023
Language Models Hallucinate, but May Excel at Fact Verification Jian Guan Jesse Dodge David Wadden Minlie Huang Hao Peng LRM HILM 40 28 0 23 Oct 2023
Tree Prompting: Efficient Task Adaptation without Fine-Tuning John X. Morris Chandan Singh Alexander M. Rush Jianfeng Gao Yuntian Deng VLM LRM 31 17 0 21 Oct 2023
Quantifying Language Models' Sensitivity to Spurious Features in Prompt Design or: How I learned to start worrying about prompt formatting Melanie Sclar Yejin Choi Yulia Tsvetkov Alane Suhr 53 308 0 17 Oct 2023
Who Are All The Stochastic Parrots Imitating? They Should Tell Us! Sagi Shaier Lawrence E Hunter K. Wense 41 3 0 16 Oct 2023
Decomposed Prompt Tuning via Low-Rank Reparameterization Yao Xiao Lu Xu Jiaxi Li Wei Lu Xiaoli Li VLM 25 6 0 16 Oct 2023
In-Context Learning for Few-Shot Molecular Property Prediction Christopher Fifty J. Leskovec Sebastian Thrun 38 5 0 13 Oct 2023
Enhancing Text-based Knowledge Graph Completion with Zero-Shot Large Language Models: A Focus on Semantic Enhancement Rui Yang Jiahao Zhu Jianping Man Li Fang Yi Zhou 24 18 0 12 Oct 2023
On the Evolution of Knowledge Graphs: A Survey and Perspective Xuhui Jiang Chengjin Xu Yinghan Shen Xun Sun Lumingyuan Tang Sai Wang Zhongwu Chen Yuanzhuo Wang Jian Guo 30 8 0 07 Oct 2023
Prompt-and-Align: Prompt-Based Social Alignment for Few-Shot Fake News Detection Jiaying Wu Xinyu Chen Haobin Yang Qi Zhao Yuhui Shi AAML 27 11 0 28 Sep 2023
Quantifying and Analyzing Entity-level Memorization in Large Language Models Zhenhong Zhou Jiuyang Xiang Chao-Yi Chen Sen Su PILM 38 8 0 30 Aug 2023
Mobile Foundation Model as Firmware Jinliang Yuan Chenchen Yang Dongqi Cai Shihe Wang Xin Yuan ... Di Zhang Hanzi Mei Xianqing Jia Shangguang Wang Mengwei Xu 40 19 0 28 Aug 2023
Semantic Consistency for Assuring Reliability of Large Language Models Harsh Raj Vipul Gupta Domenic Rosati S. Majumdar HILM 110 14 0 17 Aug 2023
A Knowledge-enhanced Two-stage Generative Framework for Medical Dialogue Information Extraction Zefa Hu Ziyi Ni Jing Shi Shuang Xu Bo Xu MedIm 43 1 0 30 Jul 2023
Pseudo Outlier Exposure for Out-of-Distribution Detection using Pretrained Transformers Jaeyoung Kim Kyuheon Jung Dongbin Na Sion Jang Eunbin Park Sungchul Choi OODD 41 6 0 18 Jul 2023
Training Models to Generate, Recognize, and Reframe Unhelpful Thoughts Mounica Maddela Megan Ung Jing Xu Andrea Madotto H. Foran Y-Lan Boureau LRM 36 21 0 06 Jul 2023
NOTABLE: Transferable Backdoor Attacks Against Prompt-based NLP Models Kai Mei Zheng Li Zhenting Wang Yang Zhang Shiqing Ma AAML SILM 37 48 0 28 May 2023
Plug-and-Play Document Modules for Pre-trained Models Chaojun Xiao Zhengyan Zhang Xu Han Chi-Min Chan Yankai Lin Zhiyuan Liu Xiangyang Li Zhonghua Li Bo Zhao Maosong Sun KELM 34 5 0 28 May 2023
Beyond Positive Scaling: How Negation Impacts Scaling Trends of Language Models Yuhui Zhang Michihiro Yasunaga Zhengping Zhou Jeff Z. HaoChen James Zou Percy Liang Serena Yeung 47 7 0 27 May 2023
RAMP: Retrieval and Attribute-Marking Enhanced Prompting for Attribute-Controlled Translation Gabriele Sarti Phu Mon Htut Xing Niu B. Hsu Anna Currey Georgiana Dinu Maria Nadejde LRM 45 10 0 26 May 2023
Hierarchical Verbalizer for Few-Shot Hierarchical Text Classification Kexi Ji Yixin Lian Jingsheng Gao Baoyuan Wang VLM 35 21 0 26 May 2023
The ACL OCL Corpus: Advancing Open Science in Computational Linguistics Shaurya Rohatgi Yanxia Qin Benjamin Aw Niranjana Unnithan MingSung Kan LMTD 20 12 0 24 May 2023
VIP5: Towards Multimodal Foundation Models for Recommendation Shijie Geng Juntao Tan Shuchang Liu Zuohui Fu Yongfeng Zhang 32 71 0 23 May 2023
Language Models with Rationality Nora Kassner Oyvind Tafjord Ashish Sabharwal Kyle Richardson Hinrich Schütze Peter Clark ReLM KELM LRM 20 15 0 23 May 2023
CTQScorer: Combining Multiple Features for In-context Example Selection for Machine Translation Aswanth Kumar Ratish Puduppully Raj Dabre Anoop Kunchukuttan 49 12 0 23 May 2023
Knowledge of Knowledge: Exploring Known-Unknowns Uncertainty with Large Language Models Alfonso Amayuelas Kyle Wong Liangming Pan Wenhu Chen Wei Wang 42 25 0 23 May 2023
Natural Language Decompositions of Implicit Content Enable Better Text Representations Alexander Miserlis Hoyle Rupak Sarkar Pranav Goel Philip Resnik AI4CE 46 12 0 23 May 2023
Language-Agnostic Bias Detection in Language Models with Bias Probing Abdullatif Köksal Omer F. Yalcin Ahmet Akbiyik M. Kilavuz Anna Korhonen Hinrich Schütze 41 1 0 22 May 2023
LM vs LM: Detecting Factual Errors via Cross Examination Roi Cohen May Hamri Mor Geva Amir Globerson HILM 41 120 0 22 May 2023
Flatness-Aware Prompt Selection Improves Accuracy and Sample Efficiency Lingfeng Shen Weiting Tan Boyuan Zheng Daniel Khashabi VLM 45 6 0 18 May 2023
Statistical Knowledge Assessment for Large Language Models Qingxiu Dong Jingjing Xu Lingpeng Kong Zhifang Sui Lei Li HILM 47 6 0 17 May 2023
Active Retrieval Augmented Generation Zhengbao Jiang Frank F. Xu Luyu Gao Zhiqing Sun Qian Liu Jane Dwivedi-Yu Yiming Yang Jamie Callan Graham Neubig RALM 30 256 0 11 May 2023
Domain Mastery Benchmark: An Ever-Updating Benchmark for Evaluating Holistic Domain Knowledge of Large Language Model--A Preliminary Release Zhouhong Gu Xiaoxuan Zhu Haoning Ye Lin Zhang Zhuozhi Xiong Zihan Li Qi He Sihang Jiang Hongwei Feng Yanghua Xiao ELM ALM 47 2 0 23 Apr 2023
An Iterative Optimizing Framework for Radiology Report Summarization with ChatGPT Chong Ma Zihao Wu Jiaqi Wang Shaochen Xu Yaonai Wei ... Tuo Zhang Dajiang Zhu Dinggang Shen Tianming Liu Xiang Li MedIm LM&MA 47 97 0 17 Apr 2023
TransHP: Image Classification with Hierarchical Prompting Wenhao Wang Yifan Sun W. Li Yi Yang 33 16 0 13 Apr 2023
Defense-Prefix for Preventing Typographic Attacks on CLIP Hiroki Azuma Yusuke Matsui VLM AAML 20 17 0 10 Apr 2023
Should ChatGPT be Biased? Challenges and Risks of Bias in Large Language Models Emilio Ferrara SILM 36 248 0 07 Apr 2023
Vita-CLIP: Video and text adaptive CLIP via Multimodal Prompting Syed Talal Wasim Muzammal Naseer Salman Khan Fahad Shahbaz Khan M. Shah VLM VPVLM 39 75 0 06 Apr 2023
Unsupervised Improvement of Factual Knowledge in Language Models Nafis Sadeq Byungkyu Kang Prarit Lamba Julian McAuley KELM 27 6 0 04 Apr 2023