检索结果-内蒙古大学图书馆

Hierarchical information matters: Text classification via tree based graph neural network

学校读者我要写书评

暂无评论

arXiv 2021年

作者： Zhang, Chong Zhu, He Peng, Xingyu Wu, Junran Xu, Ke State Key Lab of Software Development Environment Beihang University Beijing100191 China

Text classification is a primary task in natural language processing (NLP). Recently, graph neural networks (GNNs) have developed rapidly and been applied to text classification tasks. As a special kind of graph data, the tree has a simpler data structure and can provide rich hierarchical information for text classification. Inspired by the structural entropy, we construct the coding tree of the graph by minimizing the structural entropy and propose HINT, which aims to make full use of the hierarchical information contained in the text for the task of text classification. Specifically, we first establish a dependency parsing graph for each text. Then we designed a structural entropy minimization algorithm to decode the key information in the graph and convert each graph to its corresponding coding tree. based on the hierarchical structure of the coding tree, the representation of the entire graph is obtained by updating the representation of non-leaf nodes in the coding tree layer by layer. Finally, we present the effectiveness of hierarchical information in text classification. Experimental results show that HINT outperforms the state-of-the-art methods on popular benchmarks while having a simple structure and few parameters. © 2021, CC BY-NC-SA.

关键词： graph neural networks

Improving rare disease classification using imperfect knowledge graph

学校读者我要写书评

暂无评论

BMC MEDICAL INFORMATICS AND DECISION MAKING 2019年第1期19卷 238-238页

作者： Li, Xuedong Wang, Yue Wang, Dongwu Yuan, Walter Peng, Dezhong Mei, Qiaozhu Sichuan Univ Coll Comp Sci Chengdu Peoples R China Univ North Carolina Chapel Hill Sch Informat & Lib Sci Chapel Hill NC USA MobLab Inc Pasadena CA USA Univ Michigan Sch Informat Ann Arbor MI 48109 USA

Background: Accurately recognizing rare diseases based on symptom description is an important task in patient triage, early risk stratification, and target therapies. However, due to the very nature of rare diseases, the lack of historical data poses a great challenge to machine learning-based approaches. On the other hand, medical knowledge in automatically constructed knowledge graphs (KGs) has the potential to compensate the lack of labeled training examples. This work aims to develop a rare disease classification algorithm that makes effective use of a knowledge graph, even when the graph is imperfect. Method: We develop a text classification algorithm that represents a document as a combination of a "bag of words" and a "bag of knowledge terms," where a "knowledge term" is a term shared between the document and the subgraph of KG relevant to the disease classification task. We use two Chinese disease diagnosis corpora to evaluate the algorithm. The first one, HaoDaiFu, contains 51,374 chief complaints categorized into 805 diseases. The second data set, ChinaRe, contains 86,663 patient descriptions categorized into 44 disease categories. Results: On the two evaluation data sets, the proposed algorithm delivers robust performance and outperforms a wide range of baselines, including resampling, deep learning, and feature selection approaches. Both classification-based metric (macro-averaged F-1 score) and ranking-based metric (mean reciprocal rank) are used in evaluation. Conclusion: Medical knowledge in large-scale knowledge graphs can be effectively leveraged to improve rare diseases classification models, even when the knowledge graph is incomplete.

关键词： Rare disease diagnosis Knowledge graph Machine learning Text classification Extremely imbalanced data

Embedding Compression with Hashing for Efficient Representation Learning in Large-Scale graph

学校读者我要写书评

暂无评论

arXiv 2022年

作者： Yeh, Chin-Chia Michael Gu, Mengting Zheng, Yan Chen, Huiyuan Ebrahimi, Javid Zhuang, Zhongfang Wang, Junpeng Wang, Liang Zhang, Wei Visa Research United States

graph neural networks (GNNs) are deep learning models designed specifically for graph data, and they typically rely on node features as the input to the first layer. When applying such a type of network on the graph without node features, one can extract simple graph-based node features (e.g., number of degrees) or learn the input node representations (i.e., embeddings) when training the network. While the latter approach, which trains node embeddings, more likely leads to better performance, the number of parameters associated with the embeddings grows linearly with the number of nodes. It is therefore impractical to train the input node embeddings together with GNNs within graphics processing unit (GPU) memory in an end-to-end fashion when dealing with industrial-scale graph data. Inspired by the embedding compression methods developed for natural language processing (NLP) tasks, we develop a node embedding compression method where each node is compactly represented with a bit vector instead of a floating-point vector. The parameters utilized in the compression method can be trained together with GNNs. We show that the proposed node embedding compression method achieves superior performance compared to the alternatives. © 2022, CC BY.

关键词： Embeddings

Few-Shot Knowledge graph Completion based on Data Enhancement

学校读者我要写书评

暂无评论

Few-Shot Knowledge Graph Completion based on Data Enhancemen...

IEEE International Conference on Bioinformatics and Biomedicine (BIBM)

作者： Zepeng Li Peilun Geng Shuo Cao Bin Hu Gansu Provincial Key Laboratory of Wearable Computing School of Information Science and Engineering Lanzhou University Lanzhou China School of Medical Technology Beijing Institute of Technology Beijing China Chines Academy of Sciences CAS Center for Excellence in Brain Science and Intelligence Technology Shanghai Institutes for Biological Sciences Shanghai China

ISBN: (数字)9781665468190

ISBN: (纸本)9781665468206

Knowledge graphs (KGs) are widely used in various natural language processing applications. In order to expand the coverage of a KG, KG completion has attracted extensive attention. The commonly used embedding methods based on a large amount of training data can play an important role in this work. However, with few of triples, the performance of these methods will be greatly reduced. The completion of this kind of few-shot task is more challenging. In this work, we propose a method of data enhancement to increase the data quantity and solve the problem of sample shortage. Specifically, we first observe that the representation vectors of the relation in a KG are approximately subordinate to Gaussian distribution. Then we construct a Gaussian distribution for the relation of each triple in few-shot task according to the distributions of its similar relations in background graph. Further, we sample from the Gaussian distribution of each triple to expand the training data. Finally, we use an adaptive attentional network model FAAN proposed by Sheng et al. as the baseline model. Experimental results on two public datasets NELL-One and Wiki-One show that the proposed method achieves better performance.

关键词： Adaptation models Adaptive systems Biological system modeling Training data Gaussian distribution natural language processing Task analysis

Large language Models on graphs: A Comprehensive Survey

学校读者我要写书评

暂无评论

arXiv 2023年

作者： Jin, Bowen Liu, Gang Han, Chi Jiang, Meng Ji, Heng Han, Jiawei University of Illinois at Urbana-Champaign United States University of Notre Dame United States

Large language models (LLMs), such as GPT4 and LLaMA, are creating significant advancements in natural language processing, due to their strong text encoding/decoding ability and newly found emergent capability (e.g., reasoning). While LLMs are mainly designed to process pure texts, there are many real-world scenarios where text data is associated with rich structure information in the form of graphs (e.g., academic networks, and e-commerce networks) or scenarios where graph data is paired with rich textual information (e.g., molecules with descriptions). Besides, although LLMs have shown their pure text-based reasoning ability, it is underexplored whether such ability can be generalized to graphs (i.e., graph-based reasoning). In this paper, we provide a systematic review of scenarios and techniques related to large language models on graphs. We first summarize potential scenarios of adopting LLMs on graphs into three categories, namely pure graphs, text-attributed graphs, and text-paired graphs. We then discuss detailed techniques for utilizing LLMs on graphs, including LLM as Predictor, LLM as Encoder, and LLM as Aligner, and compare the advantages and disadvantages of different schools of models. Furthermore, we discuss the real-world applications of such methods and summarize open-source codes and benchmark datasets. Finally, we conclude with potential future research directions in this fast-growing field. The related source can be found at https://***/PeterGriffinJin/Awesome-language-Model-on-graphs. Copyright © 2023, The Authors. All rights reserved.

关键词： graph neural networks

Linking Theories and methods in Cognitive Sciences via Joint Embedding of the Scientific Literature: The Example of Cognitive Control

学校读者我要写书评

暂无评论

arXiv 2022年

作者： Ansarinia, Morteza Schrater, Paul Cardoso-Leite, Pedro University of Luxembourg Department of Behavioural and Cognitive Sciences Luxembourg University of Minnesota Twin Cities Depts. of Computer Science and Psychology United States

Traditionally, theory and practice of Cognitive Control are linked via literature reviews by human domain experts. This approach, however, is inadequate to track the ever-growing literature. It may also be biased, and yield redundancies and confusion. Here we present an alternative approach. We performed automated text analyses on a large body of scientific texts to create a joint representation of tasks and constructs. More specifically, 385,705 scientific abstracts were first mapped into an embedding space using a transformers-based language model. Document embeddings were then used to identify a task-construct graph embedding that grounds constructs on tasks and supports nuanced meaning of the constructs by taking advantage of constrained random walks in the graph. This joint task-construct graph embedding, can be queried to generate task batteries targeting specific constructs, may reveal knowledge gaps in the literature, and inspire new tasks and novel hypotheses. © 2022, CC BY.

关键词： natural language processing systems

Privacy-Preserving graph Machine Learning from Data to Computation: A Survey

学校读者我要写书评

暂无评论

arXiv 2023年

作者： Fu, Dongqi Bao, Wenxuan Maciejewski, Ross Tong, Hanghang He, Jingrui University of Illinois Urbana-Champaign United States Arizona State University United States

In graph machine learning, data collection, sharing, and analysis often involve multiple parties, each of which may require varying levels of data security and privacy. To this end, preserving privacy is of great importance in protecting sensitive information. In the era of big data, the relationships among data entities have become unprecedentedly complex, and more applications utilize advanced data structures (i.e., graphs) that can support network structures and relevant attribute information. To date, many graph-based AI models have been proposed (e.g., graph neural networks) for various domain tasks, like computer vision and natural language processing. In this paper, we focus on reviewing privacy-preserving techniques of graph machine learning. We systematically review related works from the data to the computational aspects. We first review methods for generating privacy-preserving graph data. Then we describe methods for transmitting privacy-preserved information (e.g., graph model parameters) to realize the optimization-based computation when data sharing among multiple parties is risky or impossible. In addition to discussing relevant theoretical methodology and software tools, we also discuss current challenges and highlight several possible future research opportunities for privacy-preserving graph machine learning. Finally, we envision a unified and comprehensive secure graph machine learning system. Copyright © 2023, The Authors. All rights reserved.

关键词： Privacy-preserving techniques

PubTrends - a scientific literature explorer 21

学校读者我要写书评

暂无评论

PubTrends - a scientific literature explorer

12th ACM Conference on Bioinformatics, Computational Biology, and Health Informatics (ACM-BCB)

作者： Shpynov, Oleg Nikolai, Kapralov JetBrains Res Biolabs St Petersburg Russia Max Planck Inst Human Cognit & Brain Sci Leipzig Leipzig Germany

ISBN: (纸本)9781450384506

With an ever-increasing number of scientific papers published each year, it becomes more difficult for researchers to explore unfamiliar or fast-growing research areas. This greatly inhibits the potential for cross-disciplinary research. When approaching a new subject, a researcher often starts with a search of relevant papers in a dedicated search engine - Google Scholar, Scopus, etc. Besides the search by title, author names, or keywords, these services also provide the user with basic statistics such as number of citations of the paper. However, it is hard to organize the information from these papers, especially with the current rise in publication numbers. For instance, regarding the latest COVID-19 epidemic, thousands of papers were published in the first *** methods [1] allow for structuring the information in papers by highlighting the most frequent keywords in the selected area. Also, the number of citations is widely used to rank papers according to their estimated scientific impact, citation and co-citation networks for groups of papers may reflect the underlying structure of the research field [2]. Papers are more likely to be co-cited if they belong to closely related research topics, and network analysis allows extracting meaningful clusters representing different scientific *** approach for structuring the information is based on the methods for natural language processing. In particular, we have shown that citation context can be used for summarization of papers and, subsequently, research topics [3].To our knowledge, no freely available tools exist that combine different approaches for structuring information in research papers in an easy-to-use web service. Most bibliometrics tools require manual preparation of papers dataset, and most of the summarization methods are lacking ready-to-use *** present PubTrends - a scientific publication exploratory tool capable of analyzing the intellectual structure of

关键词： scientific publications extractive summarization visualization citations analysis graph analysis communities detection PubMed bibliometrics

Multi-channel graph Convolution for Aspect-level Sentiment Classification of Online Student Reviews 22

学校读者我要写书评

暂无评论

Multi-channel Graph Convolution for Aspect-level Sentiment C...

proceedings of the 2022 5th International Conference on Machine Learning and natural language processing

作者： Churan Cao Peng Han Jian Qiu Dongmei Liu Li Peng Kaiqing Luo School of Physics and Telecommunications Engineering South China Normal University China School of Electronics and Information Engineering South China Normal University China

ISBN: (纸本)9781450399067

Abstract—It is important for schools and online learning platforms to mine student attitudes and opinions from student reviews so teachers know what needs to be improved. Previous studies have only focused on the semantic features of sentences, ignoring syntactic dependency relations in sentences. To address this problem, we build a graph convolutional network (GCN) on the syntactic dependency tree of sentences to exploit syntactic information for the first time in the field of online student reviews. Furthermore, in order to exploit positional relations and alleviate over-smoothing, we add relative aspect distance relations and stochastic edge elimination. On this basis, a novel multi-channel graph convolution model (MCGCN) is proposed, which constructs three GCN channels for syntactic dependency relations, relative aspect distance relations and stochastic edge elimination to extract relational features. Finally we use average pooling to fuse three relational features. Experiments on the CR23k dataset demonstrate that the overall performance of the model outperforms the state-of-the-art methods.

关键词：