Solid-SQL: Enhanced Schema-linking based In-context Learning for Robust Text-to-SQL

作     者：Liu, Geling Tan, Yunzhi Zhong, Ruichao Xie, Yuanzhen Zhao, Lingchen Wang, Qian Hu, Bo Li, Zang 

作者机构：School of Cyber Science and Engineering Wuhan University China Big Data and AI Platform Department Tencent China Key Laboratory of Aerospace Information Security and Trusted Computing Ministry of Education China 

出 版 物：《arXiv》 (arXiv)

年 卷 期：2024年

核心收录：

主　　题：Adversarial machine learning 

摘      要：Recently, large language models (LLMs) have significantly improved the performance of text-to-SQL systems. Nevertheless, many state-of-the-art (SOTA) approaches have overlooked the critical aspect of system robustness. Our experiments reveal that while LLM-driven methods excel on standard datasets, their accuracy is notably compromised when faced with adversarial perturbations. To address this challenge, we propose a robust text-to-SQL solution, called Solid-SQL, designed to integrate with various LLMs. We focus on the pre-processing stage, training a robust schema-linking model enhanced by LLM-based data augmentation. Additionally, we design a two-round, structural similarity-based example retrieval strategy for in-context learning. Our method achieves SOTA SQL execution accuracy levels of 82.1% and 58.9% on the general Spider and Bird benchmarks, respectively. Furthermore, experimental results show that Solid-SQL delivers an average improvement of 11.6% compared to baselines on the perturbed Spider-Syn, Spider-Realistic, and Dr. Spider benchmarks. Copyright © 2024, The Authors. All rights reserved.