Demystifying Prompts in Language Models via Perplexity Estimation

8 December 2022

Luke Zettlemoyer

Papers citing "Demystifying Prompts in Language Models via Perplexity Estimation"

42 / 142 papers shown

Title
The language of prompting: What linguistic properties make a prompt successful? Alina Leidinger R. Rooij Ekaterina Shutova 38 43 0 03 Nov 2023
Hint-enhanced In-Context Learning wakes Large Language Models up for knowledge-intensive tasks Yifan Wang Qingyan Guo Xinzhe Ni Chufan Shi Lemao Liu Haiyun Jiang Yujiu Yang ReLM RALM 25 8 0 03 Nov 2023
Improving Input-label Mapping with Demonstration Replay for In-context Learning Zhuocheng Gong Jiahao Liu Qifan Wang Jingang Wang Xunliang Cai Dongyan Zhao Rui Yan 24 1 0 30 Oct 2023
Unnatural language processing: How do language models handle machine-generated prompts? Corentin Kervadec Francesca Franzon Marco Baroni 23 5 0 24 Oct 2023
Bridging Information-Theoretic and Geometric Compression in Language Models Emily Cheng Corentin Kervadec Marco Baroni 34 16 0 20 Oct 2023
Quantifying Language Models' Sensitivity to Spurious Features in Prompt Design or: How I learned to start worrying about prompt formatting Melanie Sclar Yejin Choi Yulia Tsvetkov Alane Suhr 35 304 0 17 Oct 2023
Not All Demonstration Examples are Equally Beneficial: Reweighting Demonstration Examples for In-Context Learning Zhe Yang Damai Dai Peiyi Wang Zhifang Sui 39 9 0 12 Oct 2023
Dynamic Demonstrations Controller for In-Context Learning Fei Zhao Taotian Pang Zhen Wu Zheng Ma Shujian Huang Xinyu Dai 27 5 0 30 Sep 2023
Leveraging Medical Knowledge Graphs Into Large Language Models for Diagnosis Prediction: Design and Application Study Yanjun Gao Ruizhe Li John R. Caskey Dmitriy Dligach T. Miller M. Churpek Majid Afshar Dmitriy Dligach Majid Afshar LM&MA 22 5 0 28 Aug 2023
Metric-Based In-context Learning: A Case Study in Text Simplification Subhadra Vadlamannati Gözde Gül Sahin 21 1 0 27 Jul 2023
Performance of Large Language Models in a Computer Science Degree Program Tim Krüger Michael Gref ELM ALM 16 8 0 24 Jul 2023
In-Context Learning Learns Label Relationships but Is Not Conventional Learning Jannik Kossen Y. Gal Tom Rainforth 37 27 0 23 Jul 2023
InstructEval: Systematic Evaluation of Instruction Selection Methods Anirudh Ajith Chris Pan Mengzhou Xia A. Deshpande Karthik Narasimhan ELM 25 16 0 01 Jul 2023
Pushing the Limits of ChatGPT on NLP Tasks Xiaofei Sun Linfeng Dong Xiaoya Li Zhen Wan Shuhe Wang ... Jiwei Li Fei Cheng Lingjuan Lyu Fei Wu Guoyin Wang AI4MH LRM 44 29 0 16 Jun 2023
Language models are not naysayers: An analysis of language models on negation benchmarks Thinh Hung Truong Timothy Baldwin Karin Verspoor Trevor Cohn 32 54 0 14 Jun 2023
Beyond Confidence: Reliable Models Should Also Consider Atypicality Mert Yuksekgonul Linjun Zhang James Zou Carlos Guestrin 32 20 0 29 May 2023
Self-ICL: Zero-Shot In-Context Learning with Self-Generated Demonstrations Wei-Lin Chen Cheng-Kuang Wu Yun-Nung Chen Hsin-Hsi Chen 21 27 0 24 May 2023
Frugal Prompting for Dialog Models Bishal Santra Sakya Basak Abhinandan De Manish Gupta Pawan Goyal 30 2 0 24 May 2023
Improving Probability-based Prompt Selection Through Unified Evaluation and Analysis Sohee Yang Jonghyeon Kim Joel Jang Seonghyeon Ye Hyunji Lee Minjoon Seo 28 9 0 24 May 2023
In-Context Demonstration Selection with Cross Entropy Difference Dan Iter Reid Pryzant Ruochen Xu Shuohang Wang Yang Liu Yichong Xu Chenguang Zhu 23 11 0 24 May 2023
Evaluation of African American Language Bias in Natural Language Generation Nicholas Deas Jessica A. Grieser Shana Kleiner D. Patton Elsbeth Turcan Kathleen McKeown 18 26 0 23 May 2023
Active Learning Principles for In-Context Learning with Large Language Models Katerina Margatina Timo Schick Nikolaos Aletras Jane Dwivedi-Yu 30 39 0 23 May 2023
Dr.ICL: Demonstration-Retrieved In-context Learning Man Luo Xin Xu Zhuyun Dai Panupong Pasupat Mehran Kazemi Chitta Baral Vaiva Imbrasaite Vincent Zhao RALM 26 48 0 23 May 2023
CTQScorer: Combining Multiple Features for In-context Example Selection for Machine Translation Aswanth Kumar Ratish Puduppully Raj Dabre Anoop Kunchukuttan 38 12 0 23 May 2023
Enhancing Few-shot Text-to-SQL Capabilities of Large Language Models: A Study on Prompt Design Strategies Linyong Nan Yilun Zhao Weijin Zou Narutatsu Ri Jaesung Tae Ellen Zhang Arman Cohan Dragomir R. Radev LMTD 27 47 0 21 May 2023
Large Language Models can be Guided to Evade AI-Generated Text Detection Ning Lu Shengcai Liu Ruidan He Qi Wang Yew-Soon Ong Jiaheng Zhang SILM 41 43 0 18 May 2023
SatLM: Satisfiability-Aided Language Models Using Declarative Prompting Xi Ye Qiaochu Chen Işıl Dillig Greg Durrett ReLM ReCod LRM 40 62 0 16 May 2023
In-context Learning as Maintaining Coherency: A Study of On-the-fly Machine Translation Using Large Language Models Suzanna Sia Kevin Duh LRM 21 17 0 05 May 2023
Speak, Memory: An Archaeology of Books Known to ChatGPT/GPT-4 Kent K. Chang Mackenzie Cramer Sandeep Soni David Bamman RALM 145 111 0 28 Apr 2023
Large Language Model Instruction Following: A Survey of Progresses and Challenges Renze Lou Kai Zhang Wenpeng Yin ALM LRM 32 20 0 18 Mar 2023
A Theory of Emergent In-Context Learning as Implicit Structure Induction Michael Hahn Navin Goyal LRM 8 74 0 14 Mar 2023
Navigating the Grey Area: How Expressions of Uncertainty and Overconfidence Affect Language Models Kaitlyn Zhou Dan Jurafsky Tatsunori Hashimoto 21 50 0 26 Feb 2023
In-context Example Selection with Influences Nguyen Tai Eric Wong 13 48 0 21 Feb 2023
Explanation Selection Using Unlabeled Data for Chain-of-Thought Prompting Xi Ye Greg Durrett LRM ReLM 32 12 0 09 Feb 2023
A Survey on In-context Learning Qingxiu Dong Lei Li Damai Dai Ce Zheng Jingyuan Ma ... Zhiyong Wu Baobao Chang Xu Sun Lei Li Zhifang Sui ReLM AIMat 20 462 0 31 Dec 2022
Toward Human Readable Prompt Tuning: Kubrick's The Shining is a good movie, and a good prompt too? Weijia Shi Xiaochuang Han Hila Gonen Ari Holtzman Yulia Tsvetkov Luke Zettlemoyer 26 41 0 20 Dec 2022
Data Curation Alone Can Stabilize In-context Learning Ting-Yun Chang Robin Jia 21 51 0 20 Dec 2022
Extensible Prompts for Language Models on Zero-shot Language Style Customization Tao Ge Jing Hu Li Dong Shaoguang Mao Yanqiu Xia Xun Wang Si-Qing Chen Furu Wei VLM 44 6 0 01 Dec 2022
On the Relation between Sensitivity and Accuracy in In-context Learning Yanda Chen Chen Zhao Zhou Yu Kathleen McKeown He He 182 77 0 16 Sep 2022
PromptSource: An Integrated Development Environment and Repository for Natural Language Prompts Stephen H. Bach Victor Sanh Zheng-Xin Yong Albert Webson Colin Raffel ... Khalid Almubarak Xiangru Tang Dragomir R. Radev Mike Tian-Jian Jiang Alexander M. Rush VLM 225 338 0 02 Feb 2022
The Power of Scale for Parameter-Efficient Prompt Tuning Brian Lester Rami Al-Rfou Noah Constant VPVLM 280 3,848 0 18 Apr 2021
Making Pre-trained Language Models Better Few-shot Learners Tianyu Gao Adam Fisch Danqi Chen 243 1,919 0 31 Dec 2020