Improving Text-to-SQL Evaluation Methodology

23 June 2018

Catherine Finegan-Dollak

Jonathan K. Kummerfeld

Papers citing "Improving Text-to-SQL Evaluation Methodology"

50 / 175 papers shown

Title
(Un)solving Morphological Inflection: Lemma Overlap Artificially Inflates Models' Performance Omer Goldman David Guriel Reut Tsarfaty 95 28 0 12 Aug 2021
Break, Perturb, Build: Automatic Perturbation of Reasoning Paths Through Question Decomposition Mor Geva Tomer Wolfson Jonathan Berant ReLM LRM 25 21 0 29 Jul 2021
Learning Algebraic Recombination for Compositional Generalization Chenyao Liu Shengnan An Zeqi Lin Qian Liu Bei Chen Jian-Guang Lou Lijie Wen Nanning Zheng Dongmei Zhang CoGe 196 36 0 14 Jul 2021
Question Answering over Knowledge Graphs with Neural Machine Translation and Entity Linking Daniel Diomedi Aidan Hogan 16 14 0 06 Jul 2021
Draw Me a Flower: Processing and Grounding Abstraction in Natural Language R. Lachmy Valentina Pyatkin Avshalom Manevich Reut Tsarfaty 29 18 0 27 Jun 2021
Text-to-SQL in the Wild: A Naturally-Occurring Dataset Based on Stack Exchange Data Moshe Hazoom Vibhor Malik Ben Bogin 21 48 0 09 Jun 2021
Turing: an Accurate and Interpretable Multi-Hypothesis Cross-Domain Natural Language Database Interface Peng Xu Wenjie Zi H. Shahidi Ákos Kádár Keyi Tang Wei Yang Jawad Ateeq Harsh Barot Meidan Alon Yanshuai Cao 18 2 0 08 Jun 2021
One Semantic Parser to Parse Them All: Sequence to Sequence Multi-Task Learning on Semantic Parsing Datasets Marco Damonte Emilio Monti AIMat 38 6 0 08 Jun 2021
Structured Reordering for Modeling Latent Alignments in Sequence Transduction Bailin Wang Mirella Lapata Ivan Titov BDL 29 20 0 06 Jun 2021
Unlocking Compositional Generalization in Pre-trained Models Using Intermediate Representations Jonathan Herzig Peter Shaw Ming-Wei Chang Kelvin Guu Panupong Pasupat Yuan Zhang AI4CE 30 67 0 15 Apr 2021
Learning to Synthesize Data for Semantic Parsing Bailin Wang Wenpeng Yin Xi Lin Caiming Xiong 20 50 0 12 Apr 2021
Paired Examples as Indirect Supervision in Latent Decision Models Nitish Gupta Sameer Singh Matt Gardner Dan Roth 23 7 0 05 Apr 2021
FeTaQA: Free-form Table Question Answering Linyong Nan Chia-Hsuan Hsieh Ziming Mao Xi Lin Neha Verma ... Isabel Trindade Renusree Bandaru Jacob Cunningham Caiming Xiong Dragomir R. Radev LMTD 25 144 0 01 Apr 2021
Semantic Parsing to Manipulate Relational Database For a Management System M. Mushtaq 22 0 0 18 Feb 2021
An Investigation Between Schema Linking and Text-to-SQL Performance Yasufumi Taniguchi H. Nakayama Takahiro Kubo Jun Suzuki 20 5 0 03 Feb 2021
Few-Shot Semantic Parsing for New Predicates Zhuang Li Lizhen Qu Shuo Huang Gholamreza Haffari NAI 28 9 0 26 Jan 2021
Code Generation from Natural Language with Less Prior and More Monolingual Data Sajad Norouzi Keyi Tang Yanshuai Cao 20 19 0 01 Jan 2021
Iterative Utterance Segmentation for Neural Semantic Parsing Yinuo Guo Zeqi Lin Jian-Guang Lou Dongmei Zhang 44 3 0 13 Dec 2020
Tracking Interaction States for Multi-Turn Text-to-SQL Semantic Parsing Runze Wang Zhenhua Ling Jing-Bo Zhou Yu Hu 32 24 0 09 Dec 2020
Sequence-Level Mixed Sample Data Augmentation Demi Guo Yoon Kim Alexander M. Rush 25 98 0 18 Nov 2020
Character-level Representations Improve DRS-based Semantic Parsing Even in the Age of BERT Rik van Noord Antonio Toral Johan Bos 21 4 0 09 Nov 2020
"What Do You Mean by That?" A Parser-Independent Interactive Approach for Enhancing Text-to-SQL Yuntao Li Bei Chen Qian Liu Yan Gao Jian-Guang Lou Yan Zhang Dongmei Zhang 19 16 0 09 Nov 2020
Structure-Grounded Pretraining for Text-to-SQL Xiang Deng Ahmed Hassan Awadallah Christopher Meek Oleksandr Polozov Huan Sun Matthew Richardson LMTD 21 152 0 24 Oct 2020
Compositional Generalization and Natural Language Variation: Can a Semantic Parsing Approach Handle Both? Peter Shaw Ming-Wei Chang Panupong Pasupat Kristina Toutanova CoGe 27 184 0 24 Oct 2020
Did You Ask a Good Question? A Cross-Domain Question Intention Classification Benchmark for Text-to-SQL Yusen Zhang Xiangyu Dong Shuaichen Chang Tao Yu Peng Shi Rui Zhang OOD 48 15 0 23 Oct 2020
Compositional Generalization via Semantic Tagging Hao Zheng Mirella Lapata 24 34 0 22 Oct 2020
On the Potential of Lexico-logical Alignments for Semantic Parsing to SQL Queries Tianze Shi Chen Zhao Jordan L. Boyd-Graber Hal Daumé Lillian Lee 32 78 0 21 Oct 2020
DuoRAT: Towards Simpler Text-to-SQL Models Torsten Scholak Raymond Li Dzmitry Bahdanau H. D. Vries C. Pal AI4TS 35 27 0 21 Oct 2020
ColloQL: Robust Cross-Domain Text-to-SQL Over Search Queries Karthik Radhakrishnan A. Srikantan Xi Lin 24 4 0 19 Oct 2020
Knowledge Graph-based Question Answering with Electronic Health Records Junwoo Park Youngwoo Cho Haneol Lee Jaegul Choo Edward Choi 40 33 0 19 Oct 2020
Improving Compositional Generalization in Semantic Parsing I. Oren Jonathan Herzig Nitish Gupta Matt Gardner Jonathan Berant 29 63 0 12 Oct 2020
Semantic Evaluation for Text-to-SQL with Distilled Test Suites Ruiqi Zhong Tao Yu Dan Klein 11 128 0 06 Oct 2020
GraPPa: Grammar-Augmented Pre-Training for Table Semantic Parsing Tao Yu Chien-Sheng Wu Xi Lin Bailin Wang Y. Tan Xinyi Yang Dragomir R. Radev R. Socher Caiming Xiong LMTD 38 248 0 29 Sep 2020
Span-based Semantic Parsing for Compositional Generalization Jonathan Herzig Jonathan Berant ReLM LRM 17 100 0 13 Sep 2020
Photon: A Robust Cross-Domain Text-to-SQL System Jichuan Zeng Xi Lin Caiming Xiong R. Socher M. Lyu Irwin King Guosheng Lin 39 52 0 30 Jul 2020
Towards Ecologically Valid Research on Language User Interfaces H. D. Vries Dzmitry Bahdanau Christopher D. Manning 215 51 0 28 Jul 2020
Compositional Generalization in Semantic Parsing: Pre-training vs. Specialized Architectures Daniel Furrer Marc van Zee Nathan Scales Nathanael Scharli CoGe 26 113 0 17 Jul 2020
Latent Compositional Representations Improve Systematic Generalization in Grounded Question Answering Ben Bogin Sanjay Subramanian Matt Gardner Jonathan Berant ReLM OOD BDL LRM 16 28 0 01 Jul 2020
TableQA: a Large-Scale Chinese Text-to-SQL Dataset for Table-Aware SQL Generation Ningyuan Sun Xuefeng Yang Yunfeng Liu LMTD 14 34 0 10 Jun 2020
Efficient Deployment of Conversational Natural Language Interfaces over Databases Anthony Colas Trung Bui Franck Dernoncourt Moumita Sinha Doo Soon Kim 19 3 0 31 May 2020
Quda: Natural Language Queries for Visual Data Analytics Siwei Fu Kai Xiong Xiaodong Ge Siliang Tang Wei Chen Yingcai Wu 14 26 0 07 May 2020
Syntactic Question Abstraction and Retrieval for Data-Scarce Semantic Parsing Wonseok Hwang Jinyeong Yim Seunghyun Park Minjoon Seo 20 0 0 01 May 2020
A Methodology for Creating Question Answering Corpora Using Inverse Data Annotation Jan Deriu Katsiaryna Mlynchyk Philippe Schlapfer Álvaro Rodrigo D. V. Grunigen Nicolas Kaiser Kurt Stockinger Eneko Agirre Mark Cieliebak 19 15 0 16 Apr 2020
Bootstrapping a Crosslingual Semantic Parser Tom Sherborne Yumo Xu Mirella Lapata 41 25 0 06 Apr 2020
Revisit Systematic Generalization via Meaningful Learning Ning Shi Wei Ping Wei Wang Xiangyu Liu Zhouhan Lin 25 2 0 14 Mar 2020
Scalable Neural Methods for Reasoning With a Symbolic Knowledge Base William W. Cohen Haitian Sun R. A. Hofer M. Siegler 30 61 0 14 Feb 2020
Measuring Compositional Generalization: A Comprehensive Method on Realistic Data Daniel Keysers Nathanael Scharli Nathan Scales Hylke Buisman Daniel Furrer ... Tibor Tihon Dmitry Tsarkov Tianlin Li Marc van Zee Olivier Bousquet CoGe 21 348 0 20 Dec 2019
RAT-SQL: Relation-Aware Schema Encoding and Linking for Text-to-SQL Parsers Bailin Wang Richard Shin Xiaodong Liu Oleksandr Polozov Matthew Richardson 16 571 0 10 Nov 2019
Byte-Pair Encoding for Text-to-SQL Generation Samuel G. Müller Andreas Vlachos AIMat 13 2 0 20 Oct 2019
Model-based Interactive Semantic Parsing: A Unified Framework and A Text-to-SQL Case Study Ziyu Yao Yu-Chuan Su Huan Sun Wen-tau Yih 25 77 0 11 Oct 2019