检索结果-内蒙古大学图书馆

knowledge graph validation by integrating LLMs and human-in-the-loop

INFORMATION PROCESSING & MANAGEMENT 2025年第5期62卷

作者： Tsaneva, Stefani Dessi, Danilo Osborne, Francesco Sabou, Marta Vienna Univ Business & Econ Inst Data Proc & Knowledge Management Welthandelspl 1 A-1020 Vienna Austria Univ Sharjah Coll Comp & Informat Comp Sci Dept Sharjah U Arab Emirates Open Univ Knowledge Media Inst Walton Hall Milton Keynes MK7 6AA England Univ Milano Bicocca Dept Business & Law Via Bicocca Arcimboldi 8 I-20126 Milan Italy

Ensuring the quality of knowledge graphs (KGs) is crucial for the success of the intelligent applications they support. Recent advances in large language models (LLMs) have demonstrated human-level performance across various tasks, raising the question of their potential for KG validation. In this work, we explore the role of LLMs in human-centric KG validation workflows, examining different collaboration strategies between LLMs and domain experts. We propose and evaluate nine distinct approaches, ranging from fully automated validation to hybrid methods that combine expert oversight with AI assistance. These workflows are tested within a real-world KG construction pipeline used to generate the Computer Science knowledge graph (CS-KG), a large-scale resource designed to support scientometric tasks such as trend forecasting and hypothesis generation. CS-KG comprises 41 million statements represented as 350 million triples within the Computer Science domain. Our findings show that integrating LLMs into the CS-KG verification process enhances precision by 12%, improving alignment with expert-level validation. However, this comes at the cost of recall, resulting in a 5% decrease in the overall F1 score. In contrast, a hybrid approach which involves both human-in-the-loop and LLM modules, yields the best overall results, improving F1 score by 5% with minimal human involvement.

关键词： knowledge graph validation Large language models Hybrid human-AI workflows

来源：评论

学校读者我要写书评

暂无评论

Efficient knowledge graph validation via Cross-graph Representation Learning 20

Efficient Knowledge Graph Validation via Cross-Graph Represe...

引用

29th ACM International Conference on Information and knowledge Management (CIKM)

作者： Wang, Yaqing Ma, Fenglong Gao, Jing SUNY Buffalo Buffalo NY 14260 USA Penn State Univ University Pk PA 16802 USA

ISBN: (纸本)9781450368599

Recent advances in information extraction have motivated the automatic construction of huge knowledge graphs (KGs) by mining from large-scale text corpus. However, noisy facts are unavoidably introduced into KGs that could be caused by automatic extraction. To validate the correctness of facts (i.e., triplets) inside a KG, one possible approach is to map the triplets into vector representations by capturing the semantic meanings of facts. Although many representation learning approaches have been developed for knowledge graphs, these methods are not effective for validation. They usually assume that facts are correct, and thus may overfit noisy facts and fail to detect such facts. Towards effective KG validation, we propose to leverage an external human-curated KG as auxiliary information source to help detect the errors in a target KG. The external KG is built upon human-curated knowledge repositories and tends to have high precision. On the other hand, although the target KG built by information extraction from texts has low precision, it can cover new or domain-specific facts that are not in any human-curated repositories. To leverage external KG for validation, one intuitive approach is to find matched triplets between the target KG and the external KG. However, this approach can only apply to the small portion of triplets that are covered by both external and target KGs and is not useful for the validation of majority of the triplets. To tackle this challenging task, we propose a cross-graph representation learning framework, i.e., CrossVal, which can leverage an external KG to validate the facts in the target KG efficiently. This is achieved by embedding triplets based on their semantic meanings, drawing cross-KG negative samples and estimating a confidence score for each triplet based on its degree of correctness. We evaluate the proposed framework on datasets across different domains. Experimental results show that the proposed framework achieves the best pe

关键词： data cleaning knowledge graph embedding knowledge graph validation

来源：评论

学校读者我要写书评

暂无评论

TinyGenius: Intertwining Natural Language Processing with Microtask Crowdsourcing for Scholarly knowledge graph Creation 22

TinyGenius: Intertwining Natural Language Processing with Mi...

引用

22nd ACM/IEEE Joint Conference on Digital Libraries (JCDL)

作者： Oelen, Allard Stocker, Markus Auer, Soeren TIB Leibniz Informat Ctr Sci & Technol Hannover Germany

ISBN: (数字)9781450393454

ISBN: (纸本)9781665491556

As the number of published scholarly articles grows steadily each year, new methods are needed to organize scholarly knowledge so that it can be more efficiently discovered and used. Natural Language Processing (NLP) techniques are able to autonomously process scholarly articles at scale and to create machine readable representations of the article content. However, autonomous NLP methods are by far not sufficiently accurate to create a high-quality knowledge graph. Yet quality is crucial for the graph to be useful in practice. We present TinyGenius, a methodology to validate NLP-extracted scholarly knowledge statements using microtasks performed with crowdsourcing. The scholarly context in which the crowd workers operate has multiple challenges. The explainability of the employed NLP methods is crucial to provide context in order to support the decision process of crowd workers. We employed TinyGenius to populate a paper-centric knowledge graph, using five distinct NLP methods. In the end, the resulting knowledge graph serves as a digital library for scholarly articles.

关键词： Crowdsourcing Microtasks knowledge graph validation Scholarly knowledge graphs Intelligent User Interfaces

来源：评论

学校读者我要写书评

暂无评论

SCOOP All the Constraints' Flavours for Your knowledge graph 21st

SCOOP All the Constraints' Flavours for Your Knowledge Graph

引用

21st International Conference on The Semantic Web (ESWC)

作者： Duan, Xuemin Chaves-Fraga, David Derom, Olivier Dimou, Anastasia Katholieke Univ Leuven Flanders Make KULeuven Leuven AI Leuven Belgium Univ Santiago Compostela Grp Sistemas Intelixentes Santiago De Compostela Spain

ISBN: (纸本)9783031606342;9783031606359

Creating SHACL shapes for the validation of RDF graphs is a non-trivial endeavor. Automated shape extraction systems typically derive SHACL shapes from RDF graphs, and thus, their effectiveness is inherently influenced by the size and complexity of the RDF graph. However, these systems often overlook the constraints imposed by individual artifacts, although RDF graphs are often constructed by applying ontology terms to heterogeneous data. Only a few systems extract SHACL shapes from either the data schema or the ontology, leading, in either case, to limited or incomplete constraints. We propose SCOOP, a framework that exploits all artifacts associated with the construction of an RDF graph, i.e. data schemas, ontologies, and mapping rules, and integrates the SHACL shapes extracted from each artifact into a unified shapes graph. We applied our approach to real-world use cases and experimental results showed that SCOOP outperforms systems that extract SHACL shapes from RDF graphs, generating more than double the types of constraints than those systems, and effectively identifying missing and erroneous RDF triples during the validation process.

关键词： SHACL Shape integration knowledge graph validation

来源：评论

学校读者我要写书评

暂无评论

Creating and validating a scholarly knowledge graph using natural language processing and microtask crowdsourcing

引用

INTERNATIONAL JOURNAL ON DIGITAL LIBRARIES 2024年第2期25卷 273-285页

作者： Oelen, Allard Stocker, Markus Auer, Soren TIB Leibniz Informat Ctr Sci & Technol Hannover Germany

Due to the growing number of scholarly publications, finding relevant articles becomes increasingly difficult. Scholarly knowledge graphs can be used to organize the scholarly knowledge presented within those publications and represent them in machine-readable formats. Natural language processing (NLP) provides scalable methods to automatically extract knowledge from articles and populate scholarly knowledge graphs. However, NLP extraction is generally not sufficiently accurate and, thus, fails to generate high granularity quality data. In this work, we present TinyGenius, a methodology to validate NLP-extracted scholarly knowledge statements using microtasks performed with crowdsourcing. TinyGenius is employed to populate a paper-centric knowledge graph, using five distinct NLP methods. We extend our previous work of the TinyGenius methodology in various ways. Specifically, we discuss the NLP tasks in more detail and include an explanation of the data model. Moreover, we present a user evaluation where participants validate the generated NLP statements. The results indicate that employing microtasks for statement validation is a promising approach despite the varying participant agreement for different microtasks.

关键词： Crowdsourcing microtasks knowledge graph validation Scholarly knowledge graphs User interface evaluation

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：