Conventional text-to-sql research tackles the problem of solving user questions in natural language by generating the corresponding sql queries. Most of the recent works are dedicated to improving model's robustne...
详细信息
ISBN:
(纸本)9798350359329;9798350359312
Conventional text-to-sql research tackles the problem of solving user questions in natural language by generating the corresponding sql queries. Most of the recent works are dedicated to improving model's robustness and generalizability in cross- domain settings. However, model's capability in solving geography-related questions remains unexploited. In this paper we propose GS-sql, a new framework that jointly model the schema item alignment and geospatial semantics in the question. The proposed framework consists of an improved abstract syntax tree for representing spatial queries, a novel spatial entity tagging module for locating entities in the question, and a spatial semantics extraction module for determining the spatial relationship between the entities. Then we propose GeoSpatialSpider, a dataset that introduces geospatial queries, requiring model to yield spatial functions and nested sql inside functions. Finally we evaluate the proposed method on our dataset Experimental results show the effectiveness of our abstract syntax tree and GS-sql in parsing geospatial semantics while preserving traditional textto-sql capabilities.
text-to-sql systems streamline human-database interactions, improving data retrieval and decision-making. Although large language models (LLMs) can now generate sql code, challenges with generalization and uncontrolle...
详细信息
ISBN:
(纸本)9798400713293
text-to-sql systems streamline human-database interactions, improving data retrieval and decision-making. Although large language models (LLMs) can now generate sql code, challenges with generalization and uncontrolled generation hinder their use in production. text-to-sql tasks are particularly sensitive to distribution shifts, where performance declines with unfamiliar database elements or novel queries. Effective systems must maintain quality, measured in terms of generalization (correct processing of novel user requests) and error detection (identification of incorrect generations). This study empirically assesses LLM-based text-to-sql systems limitations, defining reliable production scenarios. Current contributions include a cross-lingual generalization research, study on generative model generalization abilities and the quality of selective classification for error detection risk under different distribution shifts in task of text-to-sql.
The goal of text-to-sql task is to map natural language queries into equivalent structured query languages(NL2sql). On the Wikisql dataset, the method used by the state-of-the-art models is to decouple the NL2sql task...
详细信息
The goal of text-to-sql task is to map natural language queries into equivalent structured query languages(NL2sql). On the Wikisql dataset, the method used by the state-of-the-art models is to decouple the NL2sql task into subtasks and then build a dedicated decoder for each subtask. There are some problems in this method, such as the model is too complicated, and the ability to learn the dependency between different subtasks is limited. To solve these problems, this paper innovatively introduces the sharing mechanism of multi-task learning into the NL2sql task and realizes sharing by letting different subtasks share the same decoder. Firstly, sharing decoders for different subtasks can effectively reduce the complexity of the model, and at the same time, allows different subtasks to share knowledge during the training process so that the model can better learn the dependencies between different subtasks. This paper also designed a re-weighted loss to balance the complexity of the SELECT clause and the WHERE clause. We have evaluated the method in this article on the Wikisql dataset. The experimental results show that the accuracy of the proposed model is better than state-of-the-art on the Wikisql without execution guided decoding.
NLIDB research has gained popularity recently, mainly as a means of enhancing outcomes and performance. This study makes an effort to give readers background information on how the subject has evolved recently using d...
详细信息
ISBN:
(纸本)9781665456456
NLIDB research has gained popularity recently, mainly as a means of enhancing outcomes and performance. This study makes an effort to give readers background information on how the subject has evolved recently using different text-to-sql procedures and approaches, as well as an appraisal of the advantages and disadvantages of each methodology. In contrast with past studies, this paper describes the search and selection processes and provide an overview of the complete process for each approach under review before making comparisons. The authors also evaluated the performance of each methodology against a widely recognized benchmark dataset. Along with model performance, each model was compared and assessed based on its overall structure and associated processes, such as using pre-trained language models and intermediate representations. The results of this study show that the field of text-to-sql semantic parsing has advanced significantly in recent years, as seen by the improved performance of the models under consideration. It was clear that most recent developments concentrated on the encoder side, even if each technique follows an encoder-decoder design. The imbalance opens up much room for decoder advancement in subsequent studies. Using pre-trained language models was also noteworthy for improving the models' performances;the authors will consider this for future efforts. The selection of intermediate representations, on the other hand, is wholly arbitrary.
text-to-sql aims to parse natural language problems into sql queries, which can provide a simple interface to access large databases enabling sql novices a quicker entry into databases. As the text-to-sql field is int...
详细信息
ISBN:
(纸本)9781665488679
text-to-sql aims to parse natural language problems into sql queries, which can provide a simple interface to access large databases enabling sql novices a quicker entry into databases. As the text-to-sql field is intensively studied, more and more models use GNNs to encode heterogeneous graph information in this task, and how to better obtain path information between nodes in database schema heterogeneous graphs and question-database schema heterogeneous graphs will greatly affect the effectiveness of the model parsing. Our work intends to explore the problem of solving the encoding of heterogeneous graph meta-paths in the text-to-sql task. Previous approaches usually use multi-layer GNNs to aggregate topological structure information between nodes. However, they ignored the structural information embedded at the edges and also failed to obtain nodes that are not directly connected but can provide contextual information through meta-paths. To solve the above problem, we propose Multi-Hop Relational Graph Attention Network based text-to-sql Parsing Model (MHRGATsql) for learning topological information between nodes while obtaining semantic information embedded in the edge topology. We use multi-hop attention to modify the relational graph attention network to diffuse the attention scores throughout the network, thus increasing the "receptive field" of each layer of RGAT. Experimental results on the large-scale cross-domain text-to-sql dataset Spider show that our model obtains an absolute improvement of 1.7% compared to the baseline and alleviates the over-smoothing problem in the deep network model.
Recent advances in large language models (LLMs) like Chat-GPT have led to impressive results on various natural language processing (NLP) challenges including text-to-sql task, which aims to automatically generate sql...
详细信息
ISBN:
(数字)9783031441929
ISBN:
(纸本)9783031441912;9783031441929
Recent advances in large language models (LLMs) like Chat-GPT have led to impressive results on various natural language processing (NLP) challenges including text-to-sql task, which aims to automatically generate sql queries from natural language questions. However, these languagemodels are still subject to vulnerabilities such as adversarial attacks, domain shift and lack of robustness, which can greatly affect their performance and reliability. In this paper, we conduct a comprehensive evaluation of large language models, such as ChatGPT, on their robustness in text-to-sql tasks. We assess the impact of adversarial and domain generalization perturbations on LLMs using seven datasets, five of which are popular robustness evaluation benchmarks for text-to-sql tasks and two are synthetic adversarial datasets generated by ChatGPT. Our experiments show that while LLMs exhibit promise as zero-shot text-to-sql parsers, their performances degrade under adversarial and domain generalization perturbations, with varying degrees of robustness depending on the type and level of perturbations applied. We also explore the impact of usage-related factors such as prompt design on the performance and robustness of LLMs. Our study provides insights into the limitations and potential directions for future research to enhance the performance and robustness of LLMs on text-to-sql and other NLP tasks.
text-to-sql is the task of translating a natural language query into a structured query language. Existing text-to-sql approaches focus on improving the model’s architecture while ignoring the relationship between qu...
详细信息
text-to-sql is the task of translating a natural language query into a structured query language. Existing text-to-sql approaches focus on improving the model’s architecture while ignoring the relationship between queries and table schemas and the differences in difficulty between examples in the dataset. To tackle these challenges, a two-stage curriculum learning framework for text-to-sql(TSCL-sql) is proposed in this paper. To exploit the relationship between the queries and the table schemas, a schema identification pre-training task is proposed to make the model choose the correct table schema from a set of candidates for a specific query. To leverage the differences in difficulty between examples, curriculum learning is applied to the text-to-sql task, accompanied by an automatic curriculum learning solution, including a difficulty scorer and a training scheduler. Experiments show that the framework proposed in this paper is effective.
text-to-sql task maps natural language utterances to structured queries that can be issued to a database. State-of-theart (SOTA) systems rely on finetuning large, pre-trained language models in conjunction with constr...
详细信息
ISBN:
(纸本)9798350396904
text-to-sql task maps natural language utterances to structured queries that can be issued to a database. State-of-theart (SOTA) systems rely on finetuning large, pre-trained language models in conjunction with constrained decoding applying a sql parser. On the well established Spider dataset, we begin with Oracle studies: specifically, choosing an Oracle hypothesis from a SOTA model's 10-best list, yields a 7:7% absolute improvement in both exact match (EM) and execution (EX) accuracy, showing significant potential improvements with reranking. Identifying coherence and correctness as reranking approaches, we design a model generating a query plan and propose a heuristic schema linking algorithm. Combining both approaches, with T5-Large, we obtain a consistent 1% improvement in EM accuracy, and a 2:5% improvement in EX, establishing a new SOTA for this task. Our comprehensive error studies on DEV data show the underlying difficulty in making progress on this task.
This paper introduces TURSpider, a novel Turkish text-to-sql dataset developed through human translation of the widely used Spider dataset, aimed at addressing the current lack of complex, cross-domain sql datasets fo...
详细信息
This paper introduces TURSpider, a novel Turkish text-to-sql dataset developed through human translation of the widely used Spider dataset, aimed at addressing the current lack of complex, cross-domain sql datasets for the Turkish language. TURSpider incorporates a wide range of query difficulties, including nested queries, to create a comprehensive benchmark for Turkish text-to-sql tasks. The dataset enables cross-language comparison and significantly enhances the training and evaluation of large language models (LLMs) in generating sql queries from Turkish natural language inputs. We fine-tuned several Turkish-supported LLMs on TURSpider and evaluated their performance in comparison to state-of-the-art models like GPT-3.5 Turbo and GPT-4. Our results show that fine-tuned Turkish LLMs demonstrate competitive performance, with one model even surpassing GPT-based models on execution accuracy. We also apply the Chain-of-Feedback (CoF) methodology to further improve model performance, demonstrating its effectiveness across multiple LLMs. This work provides a valuable resource for Turkish NLP and addresses specific challenges in developing accurate text-to-sql models for low-resource languages.
暂无评论