Text-to-SQL Empowered by Large Language Models: A Benchmark Evaluation

29 August 2023

Jingren Zhou

Papers citing "Text-to-SQL Empowered by Large Language Models: A Benchmark Evaluation"

50 / 50 papers shown

Title
Graph-Reward-SQL: Execution-Free Reinforcement Learning for Text-to-SQL via Graph Matching and Stepwise Reward Han Weng Boyi Liu Yuanfeng Song Dun Zeng Yingxiang Yang Yi Zhan Longjie Cui Xiaoming Yin Yang Sun 4 0 0 18 May 2025
TARGET: Benchmarking Table Retrieval for Generative Tasks Xingyu Ji Parker Glenn Aditya G. Parameswaran Madelon Hulsebos LMTD RALM 28 0 0 14 May 2025
VTS-LLM: Domain-Adaptive LLM Agent for Enhancing Awareness in Vessel Traffic Services through Natural Language Sijin Sun Liangbin Zhao Ming Deng Xiuju Fu 19 0 0 02 May 2025
Sparks of Tabular Reasoning via Text2SQL Reinforcement Learning Josefa Lia Stoisser Marc Boubnovski Martell Julien Fauqueur LMTD ReLM AI4TS LRM 88 0 0 23 Apr 2025
Think2SQL: Reinforce LLM Reasoning Capabilities for Text2SQL Simone Papicchio Simone Rossi Luca Cagliero Paolo Papotti ReLM LMTD AI4TS LRM 60 0 0 21 Apr 2025
ELT-Bench: An End-to-End Benchmark for Evaluating AI Agents on ELT Pipelines Tengjun Jin Yuxuan Zhu Daniel Kang LMTD ELM 47 0 0 07 Apr 2025
Valid Text-to-SQL Generation with Unification-based DeepStochLog Ying Jiao Luc de Raedt G. Marra NAI 40 2 0 17 Mar 2025
SQLCritic: Correcting Text-to-SQL Generation via Clause-wise Critic Jikai Chen 70 1 0 11 Mar 2025
SQL-o1: A Self-Reward Heuristic Dynamic Search Method for Text-to-SQL Shuai Lyu Haoran Luo Zhonghong Ou Yifan Zhu Xiaoran Shang Yang Qin Meina Song AI4TS LRM 65 3 0 17 Feb 2025
MultiTEND: A Multilingual Benchmark for Natural Language to NoSQL Query Translation Zhiqian Qin Yuanfeng Song Jinwei Lu Yuanwei Song Shuaimin Li Chen Zhang 48 0 0 16 Feb 2025
Rationalization Models for Text-to-SQL Gaetano Rossiello Nhan Pham Michael R. Glass Junkyu Lee Shankar Subramanian ReLM LRM 50 0 0 10 Feb 2025
ReFoRCE: A Text-to-SQL Agent with Self-Refinement, Format Restriction, and Column Exploration Minghang Deng Ashwin Ramachandran Canwen Xu Lanxiang Hu Zhewei Yao Anupam Datta Hao Zhang LMTD 137 1 0 02 Feb 2025
Is Long Context All You Need? Leveraging LLM's Extended Context for NL2SQL Yeounoh Chung Gaurav Tarlok Kakkar Yu Gan Brenton Milne Fatma Ozcan RALM 63 5 0 21 Jan 2025
Confidence Estimation for Error Detection in Text-to-SQL Systems Oleg Somov Elena Tutubalina 45 1 0 17 Jan 2025
Enhancing Table Recognition with Vision LLMs: A Benchmark and Neighbor-Guided Toolchain Reasoner Yitong Zhou Mingyue Cheng Qingyang Mao Qi Liu F. Xu Xin Li Enhong Chen LMTD 44 0 0 30 Dec 2024
Spider 2.0: Evaluating Language Models on Real-World Enterprise Text-to-SQL Workflows Fangyu Lei Jixuan Chen Yuxiao Ye Ruisheng Cao Dongchan Shin ... Caiming Xiong Ruoxi Sun Qian Liu Sida I. Wang Tao Yu LMTD 82 21 0 12 Nov 2024
Grounding Natural Language to SQL Translation with Data-Based Self-Explanations Yuankai Fan Tonghui Ren Can Huang Zhenying He Xinyu Wang LRM 47 1 0 05 Nov 2024
MSc-SQL: Multi-Sample Critiquing Small Language Models For Text-To-SQL Translation S. Gorti Ilan Gofman Zhaoyan Liu Jiapeng Wu Noël Vouitsis Guangwei Yu Jesse C. Cresswell Rasa Hosseinzadeh SyDa 58 6 0 16 Oct 2024
Accurate and Regret-aware Numerical Problem Solver for Tabular Question Answering Yuxiang Wang Jianzhong Qi Junhao Gan LMTD 53 2 0 10 Oct 2024
From Natural Language to SQL: Review of LLM-based Text-to-SQL Systems Ali Mohammadjafari Anthony Maida Raju N. Gottumukkala AI4TS 38 5 0 01 Oct 2024
E-SQL: Direct Schema Linking via Question Enrichment in Text-to-SQL Hasan Alp Caferoğlu Özgür Ulusoy 56 12 0 25 Sep 2024
Fact Finder -- Enhancing Domain Expertise of Large Language Models by Incorporating Knowledge Graphs Daniel Steinigen Roman Teucher Timm Heine Ruland Max Rudat Nicolas Flores-Herr Peter Fischer Nikola Milosevic Christopher Schymura Angelo Ziletti 36 1 0 06 Aug 2024
Lucy: Think and Reason to Solve Text-to-SQL Nina Narodytska S. Vargaftik LMTD ReLM AI4TS LRM 42 2 0 06 Jul 2024
AMBROSIA: A Benchmark for Parsing Ambiguous Questions into Database Queries Irina Saparina Mirella Lapata 57 11 0 27 Jun 2024
AnnotatedTables: A Large Tabular Dataset with Language Model Annotations Yaojie Hu Ilias Fountalis Jin Tian N. Vasiloglou LMTD 36 4 0 24 Jun 2024
Next-Generation Database Interfaces: A Survey of LLM-based Text-to-SQL Zijin Hong Zheng Yuan Qinggang Zhang Hao Chen Junnan Dong Feiran Huang Xiao Huang 77 51 0 12 Jun 2024
5W1H Extraction With Large Language Models Yang Cao Yangsong Lan Feiyan Zhai Piji Li 65 1 0 25 May 2024
LLM+Reasoning+Planning for Supporting Incomplete User Queries in Presence of APIs Sudhir Agarwal A. Sreepathy David H. Alonso Prarit Lamba LRM 60 1 0 21 May 2024
LG AI Research & KAIST at EHRSQL 2024: Self-Training Large Language Models with Pseudo-Labeled Unanswerable Questions for a Reliable Text-to-SQL System on EHRs Yongrae Jo Seongyun Lee Minju Seo Sung Ju Hwang Moontae Lee 42 3 0 18 May 2024
Open-SQL Framework: Enhancing Text-to-SQL on Open-source Large Language Models Xiaojun Chen Tianle Wang Tianhao Qiu Jianbin Qin Min Yang AI4TS 42 4 0 04 May 2024
Is Table Retrieval a Solved Problem? Exploring Join-Aware Multi-Table Retrieval Peter Baile Chen Yi Zhang Dan Roth LMTD 52 12 0 15 Apr 2024
PURPLE: Making a Large Language Model a Better SQL Writer Tonghui Ren Yuankai Fan Zhenying He Ren Huang Jiaqi Dai Can Huang Yinan Jing Kai Zhang Yifan Yang Xiaoyang Sean Wang 25 22 0 29 Mar 2024
Large Language Models(LLMs) on Tabular Data: Prediction, Generation, and Understanding -- A Survey Xi Fang Weijie Xu Fiona Anting Tan Jiani Zhang Ziqing Hu Yanjun Qi Scott Nickleach Diego Socolinsky Srinivasan H. Sengamedu Christos Faloutsos LMTD ALM 42 66 0 27 Feb 2024
BlendSQL: A Scalable Dialect for Unifying Hybrid Question Answering in Relational Algebra Parker Glenn Parag Dakle Liang Wang Preethi Raghavan ReLM 37 6 0 27 Feb 2024
From Text to CQL: Bridging Natural Language and Corpus Search Engine Luming Lu Jiyuan An Yujie Wang Liner Yang Cunliang Kong ... Shuo Wang Haozhe Lin Mingwei Fang Yaping Huang Erhong Yang 45 1 0 21 Feb 2024
Knowledge-to-SQL: Enhancing SQL Generation with Data Expert LLM Zijin Hong Zheng Yuan Hao Chen Qinggang Zhang Feiran Huang Xiao Huang 41 24 0 18 Feb 2024
Improving Demonstration Diversity by Human-Free Fusing for Text-to-SQL Dingzirui Wang Longxu Dou Xuanliang Zhang Qingfu Zhu Wanxiang Che 37 7 0 16 Feb 2024
AraSpider: Democratizing Arabic-to-SQL Ahmed Heakl Youssef Mohamed A. Zaky 38 0 0 12 Feb 2024
Using LLM to select the right SQL Query from candidates Zhenwen Li Tao Xie LLMAG 38 9 0 04 Jan 2024
MAC-SQL: A Multi-Agent Collaborative Framework for Text-to-SQL Bing Wang Changyu Ren Jian Yang Xinnian Liang Jiaqi Bai ... Zhao Yan Qian-Wen Zhang Di Yin Xing Sun Zhoujun Li 36 54 0 18 Dec 2023
LLM-SQL-Solver: Can LLMs Determine SQL Equivalence? Fuheng Zhao Lawrence Lim Ishtiyaque Ahmad D. Agrawal A. El Abbadi Amr El Abbadi 65 9 0 16 Dec 2023
Interactive Text-to-SQL Generation via Editable Step-by-Step Explanations Yuan Tian Zheng Zhang Zheng Ning Toby Jia-Jun Li Jonathan K. Kummerfeld Tianyi Zhang 43 11 0 12 May 2023
From Words to Code: Harnessing Data for Program Synthesis from Natural Language Anirudh Khatry Joyce Cahoon Jordan Henkel Shaleen Deep Venkatesh Emani ... Vu Le Mohammad Raza Sherry Shi Mukul Singh A. Tiwari 39 12 0 02 May 2023
A comprehensive evaluation of ChatGPT's zero-shot Text-to-SQL capability Aiwei Liu Xuming Hu Lijie Wen Philip S. Yu LMTD AI4MH 82 149 0 12 Mar 2023
Proton: Probing Schema Linking Information from Pre-trained Language Models for Text-to-SQL Parsing Lihan Wang Bowen Qin Binyuan Hui Bowen Li Min Yang Bailin Wang Binhua Li Fei Huang Luo Si Yongbin Li 74 40 0 28 Jun 2022
Large Language Models are Zero-Shot Reasoners Takeshi Kojima S. Gu Machel Reid Yutaka Matsuo Yusuke Iwasawa ReLM LRM 328 4,077 0 24 May 2022
Self-Consistency Improves Chain of Thought Reasoning in Language Models Xuezhi Wang Jason W. Wei Dale Schuurmans Quoc Le Ed H. Chi Sharan Narang Aakanksha Chowdhery Denny Zhou ReLM BDL LRM AI4CE 314 3,273 0 21 Mar 2022
Training language models to follow instructions with human feedback Long Ouyang Jeff Wu Xu Jiang Diogo Almeida Carroll L. Wainwright ... Amanda Askell Peter Welinder Paul Christiano Jan Leike Ryan J. Lowe OSLM ALM 339 12,003 0 04 Mar 2022
PICARD: Parsing Incrementally for Constrained Auto-Regressive Decoding from Language Models Torsten Scholak Nathan Schucher Dzmitry Bahdanau 154 375 0 10 Sep 2021
What Makes Good In-Context Examples for GPT- $3$ ? Jiachang Liu Dinghan Shen Yizhe Zhang Bill Dolan Lawrence Carin Weizhu Chen AAML RALM 275 1,312 0 17 Jan 2021