Calibrated Interpretation: Confidence Estimation in Semantic Parsing

14 November 2022

Elias Stengel-Eskin

Papers citing "Calibrated Interpretation: Confidence Estimation in Semantic Parsing"

29 / 29 papers shown

Title
Uncertainty Quantification and Confidence Calibration in Large Language Models: A Survey Xiaoou Liu Tiejin Chen Longchao Da Chacha Chen Zhen Lin Hua Wei HILM 64 3 0 20 Mar 2025
BLoB: Bayesian Low-Rank Adaptation by Backpropagation for Large Language Models Yibin Wang H. Shi Ligong Han Dimitris N. Metaxas Hao Wang BDL UQLM 110 6 0 28 Jan 2025
Confidence Estimation for Error Detection in Text-to-SQL Systems Oleg Somov Elena Tutubalina 43 1 0 17 Jan 2025
Text-to-SQL Calibration: No Need to Ask -- Just Rescale Model Probabilities Ashwin Ramachandran Sunita Sarawagi 66 2 0 23 Nov 2024
Understanding the Effect of Algorithm Transparency of Model Explanations in Text-to-SQL Semantic Parsing Daking Rai Rydia R. Weiland Kayla Margaret Gabriella Herrera Tyler H. Shaw Ziyu Yao 31 1 0 05 Oct 2024
Teaching LLMs to Abstain across Languages via Multilingual Feedback Shangbin Feng Weijia Shi Yike Wang Wenxuan Ding Orevaoghene Ahia Shuyue Stella Li Vidhisha Balachandran Sunayana Sitaram Yulia Tsvetkov 67 4 0 22 Jun 2024
LACIE: Listener-Aware Finetuning for Confidence Calibration in Large Language Models Elias Stengel-Eskin Peter Hase Mohit Bansal 35 4 0 31 May 2024
Language Model Cascades: Token-level uncertainty and beyond Neha Gupta Harikrishna Narasimhan Wittawat Jitkrittum A. S. Rawat A. Menon Sanjiv Kumar UQLM 45 42 0 15 Apr 2024
Few-Shot Recalibration of Language Models Xiang Lisa Li Urvashi Khandelwal Kelvin Guu 39 5 0 27 Mar 2024
TrustSQL: Benchmarking Text-to-SQL Reliability with Penalty-Based Scoring Gyubok Lee Woosog Chay Seonhee Cho Edward Choi LMTD 39 4 0 23 Mar 2024
Soft Self-Consistency Improves Language Model Agents Han Wang Archiki Prasad Elias Stengel-Eskin Mohit Bansal LLMAG 24 7 0 20 Feb 2024
Don't Hallucinate, Abstain: Identifying LLM Knowledge Gaps via Multi-LLM Collaboration Shangbin Feng Weijia Shi Yike Wang Wenxuan Ding Vidhisha Balachandran Yulia Tsvetkov 25 77 0 01 Feb 2024
Rephrase, Augment, Reason: Visual Grounding of Questions for Vision-Language Models Archiki Prasad Elias Stengel-Eskin Mohit Bansal ReLM LRM 28 8 0 09 Oct 2023
ReConcile: Round-Table Conference Improves Reasoning via Consensus among Diverse LLMs Justin Chih-Yao Chen Swarnadeep Saha Mohit Bansal LLMAG LRM 35 119 0 22 Sep 2023
Zero and Few-shot Semantic Parsing with Ambiguous Inputs Elias Stengel-Eskin Kyle Rawlins Benjamin Van Durme 28 14 0 01 Jun 2023
Simple Token-Level Confidence Improves Caption Correctness Suzanne Petryk Spencer Whitehead Joseph E. Gonzalez Trevor Darrell Anna Rohrbach Marcus Rohrbach 28 7 0 11 May 2023
Did You Mean...? Confidence-based Trade-offs in Semantic Parsing Elias Stengel-Eskin Benjamin Van Durme 16 5 0 29 Mar 2023
Towards Unbiased Calibration using Meta-Regularization Cheng Wang Jacek Golebiowski 26 1 0 27 Mar 2023
Navigating the Grey Area: How Expressions of Uncertainty and Overconfidence Affect Language Models Kaitlyn Zhou Dan Jurafsky Tatsunori Hashimoto 21 50 0 26 Feb 2023
The Calibration Generalization Gap Annabelle Carrell Neil Rohit Mallinar James Lucas Preetum Nakkiran UQCV 32 18 0 05 Oct 2022
ProgPrompt: Generating Situated Robot Task Plans using Large Language Models Ishika Singh Valts Blukis Arsalan Mousavian Ankit Goyal Danfei Xu Jonathan Tremblay D. Fox Jesse Thomason Animesh Garg LM&Ro LLMAG 120 622 0 22 Sep 2022
Re-Examining Calibration: The Case of Question Answering Chenglei Si Chen Zhao Sewon Min Jordan L. Boyd-Graber 59 30 0 25 May 2022
PICARD: Parsing Incrementally for Constrained Auto-Regressive Decoding from Language Models Torsten Scholak Nathan Schucher Dzmitry Bahdanau 146 374 0 10 Sep 2021
CodeT5: Identifier-aware Unified Pre-trained Encoder-Decoder Models for Code Understanding and Generation Yue Wang Weishi Wang Shafiq R. Joty S. Hoi 235 1,489 0 02 Sep 2021
Constrained Language Models Yield Few-Shot Semantic Parsers Richard Shin C. H. Lin Sam Thomson Charles C. Chen Subhro Roy Emmanouil Antonios Platanios Adam Pauls Dan Klein J. Eisner Benjamin Van Durme 292 198 0 18 Apr 2021
Reducing conversational agents' overconfidence through linguistic calibration Sabrina J. Mielke Arthur Szlam Emily Dinan Y-Lan Boureau 209 153 0 30 Dec 2020
Task-Oriented Dialogue as Dataflow Synthesis Semantic Machines Jacob Andreas J. Bufe David Burkett Charles C. Chen ... Izabela Witoszko Jason Wolfe A. Wray Yuchen Zhang Alexander Zotov AIFin 195 153 0 24 Sep 2020
Calibration of Pre-trained Transformers Shrey Desai Greg Durrett UQLM 243 289 0 17 Mar 2020
Broad-Coverage Semantic Parsing as Transduction Sheng Zhang Xutai Ma Kevin Duh Benjamin Van Durme 41 73 0 05 Sep 2019