Prompting GPT-3 To Be Reliable

17 October 2022

Jordan L. Boyd-Graber

Papers citing "Prompting GPT-3 To Be Reliable"

47 / 47 papers shown

Title
Comparing Uncertainty Measurement and Mitigation Methods for Large Language Models: A Systematic Review Toghrul Abbasli Kentaroh Toyoda Yuan Wang Leon Witt Muhammad Asif Ali Yukai Miao Dan Li Qingsong Wei UQCV 92 0 0 25 Apr 2025
CryptoPulse: Short-Term Cryptocurrency Forecasting with Dual-Prediction and Cross-Correlated Market Indicators Amit Kumar Taoran Ji 67 0 0 26 Feb 2025
Latent Space Chain-of-Embedding Enables Output-free LLM Self-Evaluation Yiming Wang Pei Zhang Baosong Yang Derek F. Wong Rui-cang Wang LRM 50 4 0 17 Oct 2024
When Context Leads but Parametric Memory Follows in Large Language Models Yufei Tao Adam Hiatt Erik Haake Antonie J. Jetter Ameeta Agrawal KELM 38 0 0 13 Sep 2024
Order Matters in Hallucination: Reasoning Order as Benchmark and Reflexive Prompting for Large-Language-Models Zikai Xie HILM LRM 61 5 0 09 Aug 2024
EventChat: Implementation and user-centric evaluation of a large language model-driven conversational recommender system for exploring leisure events in an SME context Hannes Kunstmann J. Ollier Joel Persson F. Wangenheim 37 0 0 05 Jul 2024
Know the Unknown: An Uncertainty-Sensitive Method for LLM Instruction Tuning Jiaqi Li Yixuan Tang Yi Yang 46 5 0 14 Jun 2024
Confidence Calibration and Rationalization for LLMs via Multi-Agent Deliberation Ruixin Yang Dheeraj Rajagopal S. Hayati Bin Hu Dongyeop Kang LLMAG 40 3 0 14 Apr 2024
"Flex Tape Can't Fix That": Bias and Misinformation in Edited Language Models Karina Halevy Anna Sotnikova Badr AlKhamissi Syrielle Montariol Antoine Bosselut KELM 34 3 0 29 Feb 2024
Distinguishing the Knowable from the Unknowable with Language Models Gustaf Ahdritz Tian Qin Nikhil Vyas Boaz Barak Benjamin L. Edelman 26 18 0 05 Feb 2024
Narrowing the Knowledge Evaluation Gap: Open-Domain Question Answering with Multi-Granularity Answers G. Yona Roee Aharoni Mor Geva ELM 38 11 0 09 Jan 2024
Evaluating and Mitigating Discrimination in Language Model Decisions Alex Tamkin Amanda Askell Liane Lovitt Esin Durmus Nicholas Joseph Shauna Kravec Karina Nguyen Jared Kaplan Deep Ganguli 38 66 0 06 Dec 2023
On the Potential and Limitations of Few-Shot In-Context Learning to Generate Metamorphic Specifications for Tax Preparation Software Dananjay Srinivas Rohan Das Saeid Tizpaz-Niari Ashutosh Trivedi Maria Leonor Pacheco 25 4 0 20 Nov 2023
Test-time Backdoor Mitigation for Black-Box Large Language Models with Defensive Demonstrations Wenjie Mo Jiashu Xu Qin Liu Jiong Wang Jun Yan Chaowei Xiao Muhao Chen Muhao Chen AAML 58 17 0 16 Nov 2023
Knowing What LLMs DO NOT Know: A Simple Yet Effective Self-Detection Method Yukun Zhao Lingyong Yan Weiwei Sun Guoliang Xing Chong Meng Shuaiqiang Wang Zhicong Cheng Zhaochun Ren Dawei Yin 27 35 0 27 Oct 2023
Mind the instructions: a holistic evaluation of consistency and interactions in prompt-based learning Lucas Weber Elia Bruni Dieuwke Hupkes 30 24 0 20 Oct 2023
NeMo Guardrails: A Toolkit for Controllable and Safe LLM Applications with Programmable Rails Traian Rebedea R. Dinu Makesh Narsimhan Sreedhar Christopher Parisien Jonathan Cohen KELM 19 132 0 16 Oct 2023
HuntGPT: Integrating Machine Learning-Based Anomaly Detection and Explainable AI with Large Language Models (LLMs) Tarek Ali Panos Kostakos 22 39 0 27 Sep 2023
Knowledgeable In-Context Tuning: Exploring and Exploiting Factual Knowledge for In-Context Learning J. Wang Chengyu Wang Chuanqi Tan Jun Huang Ming Gao KELM 26 4 0 26 Sep 2023
Siren's Song in the AI Ocean: A Survey on Hallucination in Large Language Models Yue Zhang Yafu Li Leyang Cui Deng Cai Lemao Liu ... Longyue Wang A. Luu Wei Bi Freda Shi Shuming Shi RALM LRM HILM 43 520 0 03 Sep 2023
Semantic Consistency for Assuring Reliability of Large Language Models Harsh Raj Vipul Gupta Domenic Rosati S. Majumdar HILM 104 14 0 17 Aug 2023
Evaluating the Ripple Effects of Knowledge Editing in Language Models Roi Cohen Eden Biran Ori Yoran Amir Globerson Mor Geva KELM 42 155 0 24 Jul 2023
Active Prompting with Chain-of-Thought for Large Language Models Shizhe Diao Pengcheng Wang Yong Lin Tong Zhang ReLM KELM LLMAG LRM 31 119 0 23 Feb 2023
REPLUG: Retrieval-Augmented Black-Box Language Models Weijia Shi Sewon Min Michihiro Yasunaga Minjoon Seo Rich James M. Lewis Luke Zettlemoyer Wen-tau Yih RALM VLM KELM 56 577 0 30 Jan 2023
Calibrated Interpretation: Confidence Estimation in Semantic Parsing Elias Stengel-Eskin Benjamin Van Durme UQLM 39 24 0 14 Nov 2022
Toward Trustworthy Neural Program Synthesis Darren Key Wen-Ding Li Kevin Ellis NAI 83 5 0 29 Sep 2022
Shortcut Learning of Large Language Models in Natural Language Understanding Mengnan Du Fengxiang He Na Zou Dacheng Tao Xia Hu KELM OffRL 31 84 0 25 Aug 2022
Red Teaming Language Models to Reduce Harms: Methods, Scaling Behaviors, and Lessons Learned Deep Ganguli Liane Lovitt John Kernion Amanda Askell Yuntao Bai ... Nicholas Joseph Sam McCandlish C. Olah Jared Kaplan Jack Clark 225 444 0 23 Aug 2022
Re-Examining Calibration: The Case of Question Answering Chenglei Si Chen Zhao Sewon Min Jordan L. Boyd-Graber 61 30 0 25 May 2022
Large Language Models are Zero-Shot Reasoners Takeshi Kojima S. Gu Machel Reid Yutaka Matsuo Yusuke Iwasawa ReLM LRM 316 4,097 0 24 May 2022
Self-Consistency Improves Chain of Thought Reasoning in Language Models Xuezhi Wang Jason W. Wei Dale Schuurmans Quoc Le Ed H. Chi Sharan Narang Aakanksha Chowdhery Denny Zhou ReLM BDL LRM AI4CE 314 3,248 0 21 Mar 2022
Chain-of-Thought Prompting Elicits Reasoning in Large Language Models Jason W. Wei Xuezhi Wang Dale Schuurmans Maarten Bosma Brian Ichter F. Xia Ed H. Chi Quoc Le Denny Zhou LM&Ro LRM AI4CE ReLM 358 8,495 0 28 Jan 2022
Fast Model Editing at Scale E. Mitchell Charles Lin Antoine Bosselut Chelsea Finn Christopher D. Manning KELM 224 341 0 21 Oct 2021
BBQ: A Hand-Built Bias Benchmark for Question Answering Alicia Parrish Angelica Chen Nikita Nangia Vishakh Padmakumar Jason Phang Jana Thompson Phu Mon Htut Sam Bowman 217 367 0 15 Oct 2021
Unsolved Problems in ML Safety Dan Hendrycks Nicholas Carlini John Schulman Jacob Steinhardt 186 273 0 28 Sep 2021
Single-dataset Experts for Multi-dataset Question Answering Dan Friedman Ben Dodge Danqi Chen MoMe 132 26 0 28 Sep 2021
Types of Out-of-Distribution Texts and How to Detect Them Udit Arora William Huang He He OODD 225 97 0 14 Sep 2021
Entity-Based Knowledge Conflicts in Question Answering Shayne Longpre Kartik Perisetla Anthony Chen Nikhil Ramesh Chris DuBois Sameer Singh HILM 245 237 0 10 Sep 2021
The Power of Scale for Parameter-Efficient Prompt Tuning Brian Lester Rami Al-Rfou Noah Constant VPVLM 280 3,848 0 18 Apr 2021
Reducing conversational agents' overconfidence through linguistic calibration Sabrina J. Mielke Arthur Szlam Emily Dinan Y-Lan Boureau 209 154 0 30 Dec 2020
Extracting Training Data from Large Language Models Nicholas Carlini Florian Tramèr Eric Wallace Matthew Jagielski Ariel Herbert-Voss ... Tom B. Brown D. Song Ulfar Erlingsson Alina Oprea Colin Raffel MLAU SILM 290 1,814 0 14 Dec 2020
It's Morphin' Time! Combating Linguistic Discrimination with Inflectional Perturbations Samson Tan Shafiq R. Joty Min-Yen Kan R. Socher 163 103 0 09 May 2020
Calibration of Pre-trained Transformers Shrey Desai Greg Durrett UQLM 243 289 0 17 Mar 2020
Scaling Laws for Neural Language Models Jared Kaplan Sam McCandlish T. Henighan Tom B. Brown B. Chess R. Child Scott Gray Alec Radford Jeff Wu Dario Amodei 234 4,469 0 23 Jan 2020
Language Models as Knowledge Bases? Fabio Petroni Tim Rocktaschel Patrick Lewis A. Bakhtin Yuxiang Wu Alexander H. Miller Sebastian Riedel KELM AI4MH 415 2,586 0 03 Sep 2019
Hypothesis Only Baselines in Natural Language Inference Adam Poliak Jason Naradowsky Aparajita Haldar Rachel Rudinger Benjamin Van Durme 190 576 0 02 May 2018
GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding Alex Jinpeng Wang Amanpreet Singh Julian Michael Felix Hill Omer Levy Samuel R. Bowman ELM 297 6,956 0 20 Apr 2018