检索结果-内蒙古大学图书馆

3rd International workshop on natural language processing for Knowledge graph Creation, NLP4KGC 2024

作者： Di Nuzzo, Dario Vakaj, Edlira Saadany, Hadeel Grishti, Eglantina Mihindukulasooriya, Nandana College of Computing Birmigham City University United Kingdom University of Tirana Tirana Albania IBM Research USA United States

This research presents a novel approach to automated competency question generation by integrating Large language Models (LLMs) with Knowledge graphs (KGs), particularly within the context of sustainability assessment standards like BREEAM. The study develops a comprehensive methodology combining natural language processing and knowledge representation to address the challenges of manual question generation in competency-based assessments. The methodology begins with text extraction from BREEAM standards, followed by preprocessing, transformation into graph documents, and the construction of a structured KG. Advanced LLMs, including GPT-4o and Mistral, are employed to generate competency questions based on entity-specific and community-focused retrieval methods. The system is rigorously evaluated using quantitative metrics such as cosine similarity scores and qualitative assessments using the "LLM-as-a-Judge"method. Results demonstrate that GPT-4 and Mistral models generate highly relevant, clear, and complex questions, highlighting the potential for scalable, domainspecific competency assessments. This research opens avenues for improving AI-driven educational technologies and personalised learning through automated, adaptive assessment tools. © 2024 Copyright for this paper by its authors.

关键词： Knowledge graph

来源：评论

学校读者我要写书评

暂无评论

Structure-Information-based Reasoning over the Knowledge graph: A Survey of methods and Applications

引用

ACM TRANSACTIONS ON KNOWLEDGE DISCOVERY FROM DATA 2024年第8期18卷 1-42页

作者： Meng, Siyuan Zhou, Jie Chen, Xuxin Liu, Yufei Lu, Fengyuan Huang, Xinli East China Normal Univ Shanghai 200062 Peoples R China

The knowledge graph (KG) is an efficient form of knowledge organization and expression, providing prior knowledge support for various downstream tasks, and has received extensive attention in natural language processing. However, existing large-scale KGs have many hidden facts that need to be discovered. How to effectively use the structure information of KG is an important research direction of knowledge reasoning. Structure-Information-based reasoning over the KG is a technique used to find the missing facts by the structure information of KG. This survey summarizes the methods and applications of Structure-Information- based reasoning and hopes to be helpful to the research in this field. first, we introduced the definition of knowledge reasoning and the conceptual description of related tasks. Then, we reviewed the methods of Structure-Information-based reasoning. Specifically, we categorized them into four representative classes: PRA-based reasoning, Path-Embedding-based reasoning, RL-based reasoning, and GNN-based reasoning. We compared the motivations and details between practices in the same category. After that, we described the application of Structure-Information-based knowledge reasoning in the KG Completion, Question Answering System, Recommendation System, and other fields. Finally, we discussed the future research directions of Structure-Information-based reasoning.

关键词： General and reference Surveys and overviews Computing methodologies Knowledge representation and reasoning

来源：评论

学校读者我要写书评

暂无评论

Toward Auto-Modeling of Formal Verification for NextG Protocols: A Multimodal Cross- and Self-Attention Large language Model Approach

引用

IEEE ACCESS 2024年 12卷 27858-27869页

作者： Yang, Jingda Wang, Ying Stevens Inst Technol Sch Syst & Enterprises Hoboken 07030 NJ USA

This paper introduces Auto-modeling of Formal Verification with Real-world Prompting for 5G and NextG protocols (AVRE), a novel system designed for the formal verification of Next Generation (NextG) communication protocols, addressing the increasing complexity and scalability challenges in network protocol design and verification. Utilizing Large language Models (LLMs), AVRE transforms protocol descriptions into dependency graphs and formal models, efficiently resolving ambiguities and capturing design intent. The system integrates a transformer model with LLMs to autonomously establish quantifiable dependency relationships through cross- and self-attention mechanisms. Enhanced by iterative feedback from the HyFuzz experimental platform, AVRE significantly advances the accuracy and relevance of formal verification in complex communication protocols, offering a groundbreaking approach to validating sophisticated communication systems. We compare CAL's performance with state-of-the-art LLM-based models and traditional time sequence models, demonstrating its superiority in accuracy and robustness, achieving an accuracy of 95.94% and an AUC of 0.98. This NLP-based approach enables, for the first time, the creation of exploits directly from design documents, making remarkable progress in scalable system verification and validation.

关键词： Protocols Formal verification 5G mobile communication natural languages Iterative methods Transformers Complexity theory natural language processing cross-attention self-attention natural language protocol formal flow graph

来源：评论

学校读者我要写书评

暂无评论

CoverICL: Selective Annotation for In-Context Learning via Active graph Coverage

CoverICL: Selective Annotation for In-Context Learning via A...

引用

2024 Conference on Empirical methods in natural language processing, EMNLP 2024

作者： Mavromatis, Costas Srinivasan, Balasubramaniam Shen, Zhengyuan Zhang, Jiani Rangwala, Huzefa Faloutsos, Christos Karypis, George Amazon Web Services United States University of Minnesota United States

ISBN: (纸本)9798891761643

In-context learning (ICL) adapts Large language Models (LLMs) to new tasks, without requiring any parameter updates, but few annotated examples as input. In this work, we investigate selective annotation for ICL, where there is a limited budget for annotating examples, similar to low-budget active learning (AL). Although uncertainty-based selection is unreliable with few annotated data, we present COVERICL, an adaptive graph-based selection algorithm, that effectively incorporates uncertainty sampling into selective annotation for ICL. first, COVERICL builds a nearest-neighbor graph based on the semantic similarity between candidate ICL examples. Then, COVERICL employs uncertainty estimation by the LLM to identify hard examples for the task. Selective annotation is performed over the active graph of the hard examples, adapting the process to the particular LLM used and the task tackled. COVERICL selects the most representative examples by solving a Maximum Coverage problem, approximating diversity-based sampling. Extensive experiments on ten datasets and seven LLMs show that, by incorporating uncertainty via coverage on the active graph, COVERICL (1) outperforms existing AL methods for ICL by 2-4.6% accuracy points, (2) is up to 2× more budget-efficient than SOTA methods for low-budget AL, and (3) generalizes better across tasks compared to non-graph alternatives. © 2024 Association for Computational Linguistics.

关键词： Active learning

来源：评论

学校读者我要写书评

暂无评论

An Evaluation Framework for Mapping News Headlines to Event Classes in a Knowledge graph 6

An Evaluation Framework for Mapping News Headlines to Event ...

引用

6th workshop on Challenges and Applications of Automated Extraction of Socio-Political Events from Text, CASE 2023

作者： Mbouadeu, Steve Fonin Lorenzo, Martin Barker, Ken Hassanzadeh, Oktie St. John’s University United States IBM Research United States

ISBN: (纸本)9789544520892

Mapping ongoing news headlines to event-related classes in a rich knowledge base can be an important component in a knowledge-based event analysis and forecasting solution. In this paper, we present a methodology for creating a benchmark dataset of news headlines mapped to event classes in Wikidata, and resources for the evaluation of methods that perform the mapping. We use the dataset to study two classes of unsupervised methods for this task: 1) adaptations of classic entity linking methods, and 2) methods that treat the problem as a zero-shot text classification problem. For the first approach, we evaluate off-the-shelf entity linking systems. For the second approach, we explore a) pre-trained natural language inference (NLI) models, and b) pre-trained large generative language models. We present the results of our evaluation, lessons learned, and directions for future work. The dataset and scripts for evaluation are made publicly available. © CASE 2023 - proceedings of the 6th workshop on Challenges and Applications of Automated Extraction of Socio-Political Events from Text, associated with 14th International Conference on Recent Advances in natural language processing, RANLP 2023.

关键词： Mapping

来源：评论

学校读者我要写书评

暂无评论

VISPool: Enhancing Transformer Encoders with Vector Visibility graph Neural Networks 62

VISPool: Enhancing Transformer Encoders with Vector Visibili...

引用

62nd Annual Meeting of the Association-for-Computational-Linguistics (ACL) / Student Research workshop (SRW)

作者： Alikasifoglu, Tuna Aras, Arda Can Koc, Aykut Bilkent Univ Dept Elect & Elect Engn Ankara Turkiye Bilkent Univ UMRAM Ankara Turkiye

ISBN: (纸本)9798891760998

The emergence of transformers has revolutionized natural language processing (NLP), as evidenced in various NLP tasks. While graph neural networks (GNNs) show recent promise in NLP, they are not standalone replacements for transformers. Rather, recent research explores combining transformers and GNNs. Existing GNN-based approaches rely on static graph construction methods requiring excessive text processing, and most of them are not scalable with the increasing document and word counts. We address these limitations by proposing a novel dynamic graph construction method for text documents based on vector visibility graphs (VVGs) generated from transformer output. Then, we introduce visibility pooler (VISPool), a scalable model architecture that seamlessly integrates VVG convolutional networks into transformer pipelines. We evaluate the proposed model on the General language Understanding Evaluation (GLUE) benchmark datasets. VISPool outperforms the baselines with less trainable parameters, demonstrating the viability of the visibility-based graph construction method for enhancing transformers with GNNs.(1)

关键词： graph neural networks

来源：评论

学校读者我要写书评

暂无评论

Universal embedding for pre-trained models and data bench

引用

NEUROCOMPUTING 2025年 619卷

作者： Cho, Namkyeong Cho, Taewon Shin, Jaesun Jeon, Eunjoo Lee, Taehee Pohang Univ Sci & Technol POSTECH Ctr Math Machine Learning & its Applicat CM2LA Dept Math Pohang 37673 Gyeongbuk South Korea Samsung SDS 125 Olymp Ro 35 Gil Seoul 05510 South Korea

The transformer architecture has shown significant improvements in the performance of various natural language processing (NLP) tasks. One of the great advantages of transformer-based model is that they allow for the addition of an extra layer to a pre-trained model (PTM) and fine-tuning, rather than requiring the development of a separate architecture for each task. This approach has provided great promising performance in NLP tasks. Therefore, selecting an appropriate PTM from the model zoo, such as Hugging Face, becomes a crucial task. Despite the importance of PTM selection, it still requires further investigation. The main challenge in PTM selection for NLP tasks is the lack of a publicly available benchmark to evaluate model performance for each task and dataset. To address this challenge, we introduce the first public data benchmark to evaluate the performance of popular transformer-based models on diverse ranges of NLP tasks. Furthermore, we propose graph representations of transformer-based models with node features that represent the matrix weight on each layer. Empirical results demonstrate that our proposed graph neural network (GNN) model outperforms existing PTM selection methods.

关键词： Transfer learning Pretrained models graph neural networks

来源：评论

学校读者我要写书评

暂无评论

The Second workshop on Knowledge-Augmented methods for natural language processing 23

The Second Workshop on Knowledge-Augmented Methods for Natur...

引用

29th ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD)

作者： Yu, Wenhao Tong, Lingbo Shi, Weijia Peng, Nanyun Jiang, Meng Univ Notre Dame Notre Dame IN 46556 USA Univ Washington Seattle WA 98195 USA Univ Calif Los Angeles Los Angeles CA USA

ISBN: (纸本)9798400701030

language models are being developed and deployed in many applications, "small"-scale and large-scale, generic and specialized, text-only and multimodal, etc. Meanwhile, the missingness of important knowledge causes limitations and safety challenges. The knowledge includes commonsense, world facts, domain expertise, personalization, and especially the unique patterns that need to be discovered from big data applications. Training and inference processes of the language models can be and should be augmented with the knowledge. The first KnowledgeNLP at AAAI 2023 attracted scientists on knowledge augmentation methods towards higher language intelligence. This workshop offers a broad platform to share ideas and discuss various topics, such as (1) synergy between knowledge and language model, (2) scalable architectures that integrate NLP, knowledge graph, and graph learning technologies, (3) KnowledgeNLP for e-commerce, education, and healthcare, (4) human factors and social good in KnowledgeNLP.

关键词： knowledge augmentation question answering text generation knowledge graph language model

来源：评论

学校读者我要写书评

暂无评论

Multi-level feature subgraph aggregation graph neural network

引用

INFORMATION FUSION 2025年 121卷

作者： Xie, Chengxin Song, Qiya Huang, Jingui Chen, Wei Pan, Liangrui Hunan Normal Univ Coll Informat Sci & Engn Changsha 410000 Peoples R China Hunan Normal Univ Hunan Prov Key Lab Intelligent Comp & Language Inf Changsha 410000 Peoples R China Hebei Univ Architecture Coll Informat Engn Zhangjiakou 075000 Peoples R China Hunan Univ Coll Comp Sci & Elect Engn Changsha 410000 Peoples R China

Heterogeneous graph Convolutional Networks (HGCNs) have become a pivotal technology for pattern mining in heterogeneous graphs. However, existing methods for heterogeneous graphs often overlook the distinct interactions between homogeneous and heterogeneous neighbors. Additionally, they fail to capture high-level features typically present in graphs, such as global topology, semantics, and attributes. To address these limitations, we propose a general and precise multi-level feature subgraph aggregation model for heterogeneous graphs (SAGNN). The SAGNN model first applies nonlinear transformations to map features of various node types into latent subspaces. A meta-path-based subgraph generation module is then used to decompose the heterogeneous graph into both homogeneous and heterogeneous subgraphs. To enable effective aggregation of node information within these subgraphs, we introduce a innovative subgraph aggregation strategy. Finally, weighted coefficients are applied to integrate node information from both homogeneous and heterogeneous subgraphs, ensuring the comprehensive fusion of multi-level features. Extensive experimental results demonstrate that the proposed SAGNN effectively extracts and fuses diverse features within heterogeneous graph structures, thereby improving performance in tasks such as node classification and clustering.

关键词： Heterogeneous graph Homogeneous graph Multi-level features Subgraph aggregation

来源：评论

学校读者我要写书评

暂无评论

Knowledge graph Enhanced Large language Model Editing

Knowledge Graph Enhanced Large Language Model Editing

引用

2024 Conference on Empirical methods in natural language processing, EMNLP 2024

作者： Zhang, Mengqi Ye, Xiaotian Liu, Qiang Ren, Pengjie Wu, Shu Chen, Zhumin School of Computer Science and Technology Shandong University China School of Computer Science Beijing University of Posts and Telecommunications China Institute of Automation Chinese Academy of Sciences China

ISBN: (纸本)9798891761643

Large language models (LLMs) are pivotal in advancing natural language processing (NLP) tasks, yet their efficacy is hampered by inaccuracies and outdated knowledge. Model editing emerges as a promising solution to address these challenges. However, existing editing methods struggle to track and incorporate changes in knowledge associated with edits, which limits the generalization ability of post-edit LLMs in processing edited knowledge. To tackle these problems, we propose a novel model editing method that leverages knowledge graphs for enhancing LLM editing, namely GLAME. Specifically, we first utilize a knowledge graph augmentation module to uncover associated knowledge that has changed due to editing, obtaining its internal representations within LLMs. This approach allows knowledge alterations within LLMs to be reflected through an external graph structure. Subsequently, we design a graph-based knowledge edit module to integrate structured knowledge into the model editing. This ensures that the updated parameters reflect not only the modifications of the edited knowledge but also the changes in other associated knowledge resulting from the editing process. Comprehensive experiments conducted on GPT-J and GPT-2 XL demonstrate that GLAME significantly improves the generalization capabilities of post-edit LLMs in employing edited knowledge. © 2024 Association for Computational Linguistics.

关键词： Knowledge graph

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：