检索结果-内蒙古大学图书馆

Multi-Level Representation Learning for Chinese Medical Entity Recognition: Model Development and Validation

JMIR MEDICAL INFORMATICS 2020年第5期8卷 e17637页

作者： Zhang, Zhichang Zhu, Lin Yu, Peilin Univ Northwest Normal Coll Comp Sci & Engn 967 Arming East Rd Lanzhou Peoples R China

Background: Medical entity recognition is a key technology that supports the development of smart medicine. Existing methods on English medical entity recognition have undergone great development, but their progress in the Chinese language has been slow. Because of limitations due to the complexity of the Chinese language and annotated corpora, these methods are based on simple neural networks, which cannot effectively extract the deep semantic representations of electronic medical records (EMRs) and be used on the scarce medical corpora. We thus developed a new Chinese EMR (CEMR) dataset with six types of entities and proposed a multi-level representation learning model based on Bidirectional Encoder Representation from Transformers (BERT) for Chinese medical entity recognition. Objective: This study aimed to improve the performance of the language model by having it learn multi-level representation and recognize Chinese medical entities. methods: In this paper, the pretraining language representation model was investigated;utilizing information not only from the final layer but from intermediate layers was found to affect the performance of the Chinese medical entity recognition task. Therefore, we proposed a multi-level representation learning model for entity recognition in Chinese EMRs. Specifically, we first used the BERT language model to extract semantic representations. Then, the multi-head attention mechanism was leveraged to automatically extract deeper semantic information from each layer. Finally, semantic representations from multi-level representation extraction were utilized as the final semantic context embedding for each token and we used softmax to predict the entity tags. Results: The best F1 score reached by the experiment was 82.11% when using the CEMR dataset, and the F1 score when using the CCKS (China Conference on Knowledge graph and Semantic Computing) 2018 benchmark dataset further increased to 83.18%. Various comparative experiments show

关键词： medical entity recognition multi-level representation learning Chinese natural language processing electronic medical records multi-head attention mechanism

来源：评论

学校读者我要写书评

暂无评论

Point Cloud Geometry Coding with Relational Neighborhood Self-Attention

Point Cloud Geometry Coding with Relational Neighborhood Sel...

引用

IEEE workshop on Multimedia Signal processing

作者： Mohammadreza Ghafari André F. R. Guarda Nuno M. M. Rodrigues Fernando Pereira Instituto Superior Técnico - Universidade de Lisboa Instituto de Telecomunicações Lisbon Portugal Instituto de Telecomunicações Lisbon Portugal ESTG-Politécnico de Leiria Instituto de Telecomunicações Lisbon Portugal

ISBN: (数字)9798350387254

ISBN: (纸本)9798350387261

In the ever-evolving landscape of deep learning, attention models have contributed to boost the performance in diverse fields such as computer vision and natural language processing. Following this trend, this paper proposes a novel Relational Neighborhood Self-Attention (RNSA) model, specifically designed for Point Cloud (PC) geometry coding to be integrated in the emerging learning-based JPEG PCC standard. The RNSA model proposes three new methods: first, to effectively learn correlations between the points by capturing the relational features and positions of neighboring points; second, to address the inefficiencies of conventional dot product attention, a novel Relational Scoring method to generate an attention map able to capture both linear and non-linear relationships between points and their neighbors is adopted; third, the created attention maps are normalized by Sparsemax instead of Softmax to generate sparse probabilities and assigns higher scores to the most important neighbors while marginalizing the less significant ones. Experimental results show that the proposed attention model achieves around 8% gains in both BD-Rate PSNR Dl and PSNR D2 compared to the baseline codec, i.e., JPEG PCC, while adding a small number of model parameters to JPEG PCC.

关键词： Point cloud compression Geometry Deep learning Computational modeling Transform coding Signal processing Market research Encoding natural language processing Standards

来源：评论

学校读者我要写书评

暂无评论

FF-PRec: A Feature Fusion-based Paper Recommendation Method on Academic Big Data of Heterogeneous Network

FF-PRec: A Feature Fusion-Based Paper Recommendation Method ...

引用

Computer Information Science and Artificial Intelligence (CISAI), International Conference on

作者： Yongkang Du Weilong Ding Ying Liang School of Information Science and Technology North China University of Technology Beijing China Research Center for Ubiquitous Computing Systems Institute of Computing Technology Chinese Academy of Science Beijing China

ISBN: (纸本)9781665406932

With the development of information technology, academic data has increased dramatically. Nowadays, recommendation algorithm is widely used for scholars to find useful information from massive data. However, due to the discrete data and complex semantic relationship in the academic network, existing recommendation algorithms have limitations in feature extraction and data sparsity, which bring negative impacts on recommendation accuracy and personalization. In order to solve the above problems, we propose FF-PRec, a paper recommendation method based on feature fusion in the academic network. first, FF-PRec implements graph representation learning and a natural language processing tool to extract network features and text features respectively. Two types of features are combined as the representation vectors of scholars and papers. Second, a meta-path is designed based on prior knowledge to guide semantic information extraction. To validate our proposed method, we conducted experiments on the AMiner dataset. The experimental results indicate that FF-PRec performs better than traditional methods in paper recommendation tasks and shows high accuracy and correlation.

关键词： Representation learning Information science Correlation Semantics Feature extraction Information retrieval natural language processing

来源：评论

学校读者我要写书评

暂无评论

Comparing approaches to dravidian language identification

arXiv

引用

arXiv 2021年

作者： Jauhiainen, Tommi Ranasinghe, Tharindu Zampieri, Marcos University of Helsinki Finland University of Wolverhampton United Kingdom Rochester Institute of Technology United States

This paper describes the submissions by team HWR to the Dravidian language Identification (DLI) shared task organized at VarDial 2021 workshop. The DLI training set includes 16,674 YouTube comments written in Roman script containing code-mixed text with English and one of the three South Dravidian languages: Kannada, Malayalam, and Tamil. We submitted results generated using two models, a Naive Bayes classifier with adaptive language models, which has shown to obtain competitive performance in many language and dialect identification tasks, and a transformer-based model which is widely regarded as the state-of-the-art in a number of NLP tasks. Our first submission was sent in the closed submission track using only the training set provided by the shared task organisers, whereas the second submission is considered to be open as it used a pretrained model trained with external data. Our team attained shared second position in the shared task with the submission based on Naive Bayes. Our results reinforce the idea that deep learning methods are not as competitive in language identification related tasks as they are in many other text classification tasks. © 2021, CC BY.

关键词： natural language processing systems

来源：评论

学校读者我要写书评

暂无评论

LASAGNE: Locality and Structure Aware graph Node Embedding

arXiv

引用

arXiv 2017年

作者： Faerman, Evgeniy Borutta, Felix Fountoulakis, Kimon Mahoney, Michael W. Ludwig-Maximilians-Universitat Munchen University of California at Berkeley

Recent work has attempted to identify structure in social and information graphs by using the following approach: first, use random walk methods to explore the neighborhood of a node;second, use ideas from natural language processing to use this neighborhood information to learn vector representations of these nodes reflecting properties of the graph. Informally, the idea is that if a node is a member of a meaningful cluster or community, then the vector representation should be higher-quality, thereby leading to improved learning. In this paper, we identify and remedy an important shortcoming of this approach. In particular, we show that the performance of existing methodologies depends strongly on the structural properties of the graph, e.g., the size of the graph, whether the graph has a at or upward-sloping Network Community Profile (NCP), whether the graph is expander-like, whether the classes of interest are more k-core-like or more peripheral, etc. For larger graphs with at NCPs that are strongly expander-like, existing methods lead to random walks that expand rapidly, touching many dissimilar nodes, thereby leading to lower-quality vector representations that are less useful for downstream tasks. based on our findings, we propose Lasagne, a methodology to learn locality and structure aware graph node embeddings in an unsupervised way. Rather than relying on global random walks or neighbors within fixed hop distances, Lasagne exploits strongly local Approximate Personalized PageRank stationary distributions to more precisely engineer local information into node embeddings. This leads, in particular, to more meaningful and more useful vector representations of nodes in poorly-structured graphs. We show that Lasagne leads to significant improvement in downstream multi-label classification for larger graphs with at NCPs, that it is comparable for smaller graphs with upward-sloping NCPs, and that is comparable to existing methods for link prediction tasks. Copyri

关键词： graphic methods

来源：评论

学校读者我要写书评

暂无评论

Time-Aware Multiway Adaptive Fusion Network for Temporal Knowledge graph Question Answering

Time-Aware Multiway Adaptive Fusion Network for Temporal Kno...

引用

International Conference on Acoustics, Speech, and Signal processing (ICASSP)

作者： Yonghao Liu Di Liang Fang Fang Sirui Wang Wei Wu Rui Jiang Centre for Natural Language Processing Meituan Inc Beijing China Department of Automation Tsinghua University Beijing China

Knowledge graphs (KGs) have received increasing attention due to its wide applications on natural language processing. However, its use case on temporal question answering (QA) has not been well-explored. Most of existing methods are developed based on pre-trained language models, which might not be capable to learn temporal-specific presentations of entities in terms of temporal KGQA task. To alleviate this problem, we propose a novel Time-aware Multiway Adaptive (TMA) fusion network. Inspired by the step-by-step reasoning behavior of humans. For each given question, TMA first extracts the relevant concepts from the KG, and then feeds them into a multiway adaptive module to produce a temporal-specific representation of the question. This representation can be incorporated with the pre-trained KG embedding to generate the final prediction. Empirical results verify that the proposed model achieves better performance than the state-of-the-art models in the benchmark dataset. Notably, the Hits@1 and Hits@10 results of TMA on the CronQuestions dataset’s complex questions are absolutely improved by 24% and 10% compared to the best-performing baseline. Furthermore, we also show that TMA employing an adaptive fusion mechanism can provide interpretability by analyzing the proportion of information in question representations.

关键词： Adaptation models Adaptive systems Fuses Knowledge graphs Signal processing Question answering (information retrieval) Robustness

来源：评论

学校读者我要写书评

暂无评论

Back to the Future: Towards Explainable Temporal Reasoning with Large language Models

arXiv

引用

arXiv 2023年

作者： Yuan, Chenhan Xie, Qianqian Huang, Jimin Ananiadou, Sophia The University of Manchester Manchester United Kingdom Wuhan University Wuhan China

Temporal reasoning is a crucial natural language processing (NLP) task, providing a nuanced understanding of time-sensitive contexts within textual data. Although recent advancements in Large language Models (LLMs) have demonstrated their potential in temporal reasoning, the predominant focus has been on tasks such as temporal expression detection, normalization, and temporal relation extraction. These tasks are primarily designed for the extraction of direct and past temporal cues from given contexts and to engage in simple reasoning processes. A significant gap remains when considering complex reasoning tasks such as event forecasting, which requires multi-step temporal reasoning on events and prediction on the future timestamp. Another notable limitation of existing methods is their incapability to provide an illustration of their reasoning process for explaining their prediction, hindering explainability. In this paper, we introduce the first task of explainable temporal reasoning, to predict an event’s occurrence at a future timestamp based on context which requires multiple reasoning over multiple events, and subsequently provide a clear explanation for their prediction. Our task offers a comprehensive evaluation of both the LLMs’ complex temporal reasoning ability, the future event prediction ability, and explainability—a critical attribute for AI applications. To support this task, we present the first multi-source instruction-tuning dataset of explainable temporal reasoning (ExpTime) with 26k derived from the temporal knowledge graph datasets and their temporal reasoning paths, using a novel knowledge-graph-instructed-generation strategy. based on the dataset, we propose the first open-source LLM series TimeLlaMA based on the foundation LLM LlaMA2, with the ability of instruction following for explainable temporal reasoning. We compare the performance of our method and a variety of LLMs, where our method achieves the state-of-the-art performance of temporal

关键词： Forecasting

来源：评论

学校读者我要写书评

暂无评论

Efficient Neural Architecture Search for End-to-End Speech Recognition Via Straight-Through Gradients

Efficient Neural Architecture Search for End-to-End Speech R...

引用

IEEE Spoken language Technology workshop

作者： Huahuan Zheng Keyu An Zhijian Ou Speech Processing and Machine Intelligence (SPMI) Lab Tsinghua University China

ISBN: (数字)9781728170664

ISBN: (纸本)9781728170671

Neural Architecture Search (NAS), the process of automating architecture engineering, is an appealing next step to advancing end-to-end Automatic Speech Recognition (ASR), replacing expert-designed networks with learned, task-specific architectures. In contrast to early computational-demanding NAS methods, recent gradient-based NAS methods, e.g., DARTS (Differentiable ARchiTecture Search), SNAS (Stochastic NAS) and ProxylessNAS, significantly improve the NAS efficiency. In this paper, we make two contributions. first, we rigorously develop an efficient NAS method via Straight-Through (ST) gradients, called ST-NAS. Basically, ST-NAS uses the loss from SNAS but uses ST to back-propagate gradients through discrete variables to optimize the loss, which is not revealed in ProxylessNAS. Using ST gradients to support sub-graph sampling is a core element to achieve efficient NAS beyond DARTS and SNAS. Second, we successfully apply ST-NAS to end-to-end ASR. Experiments over the widely benchmarked 80-hour WSJ and 300-hour Switchboard datasets show that the ST-NAS induced architectures significantly outperform the human-designed architecture across the two datasets. Strengths of ST-NAS such as architecture transferability and low computation cost in memory and time are also reported.

关键词： Conferences Computer architecture Switches Benchmark testing Computational efficiency Task analysis Automatic speech recognition

来源：评论

学校读者我要写书评

暂无评论

Keyphrase Extraction for Technical language processing

引用

JOURNAL OF RESEARCH OF THE NATIONAL INSTITUTE OF STANDARDS AND TECHNOLOGY 2022年 126卷 126053页

作者： Dima, Alden Massey, Aaron NIST Informat Syst Grp Gaithersburg MD 20899 USA Univ Maryland Baltimore Cty Software Engn Baltimore MD 21250 USA

Keyphrase extraction is an important facet of annotation tools that offer the provision of the metadata necessary for technical language processing (TLP). Because TLP imposes additional requirements on typical natural language processing (NLP) methods, we examined TLP keyphrase extraction through the lens of a hypothetical toolkit which consists of a combination of text features and classifiers suitable for use in low-resource TLP applications. We compared two approaches for keyphrase extraction: The first which applied our toolkit-based methods that used only distributional features of words and phrases, and the second was the Maui automatic topic indexer, a well-known academic method. Performance was measured against two collections of technical literature: 1153 articles from Journal of Chemical Thermodynamics (JCT) curated by the National Institute of Standards and Technology Thermodynamics Research Center (TRC) and 244 articles from Task 5 of the workshop on Semantic Evaluation (SemEval). Both collections have author-provided keyphrases available;the SemEval articles also have reader-provided keyphrases. Our findings indicate that our toolkit approach was competitive with Maui when author-provided keyphrases were first removed from the text. For the TRC-JCT articles, the Maui automatic topic indexer reported an F-measure of 29.4 % while our toolkit approach obtained an F-measure of 28.2 %. For the SemEval articles, our toolkit approach using a Naive Bayes classifier resulted in an F-measure of 20.8 %, which outperformed Maui's F-measure of 18.8 %.

关键词： keyphrase extraction technical articles technical language processing

来源：评论

学校读者我要写书评

暂无评论

Generative non-autoregressive unsupervised keyphrase extraction with neural topic modeling

引用

ENGINEERING APPLICATIONS OF ARTIFICIAL INTELLIGENCE 2023年第1期120卷

作者： Zhu, Xun Lou, Yinxia Zhao, Jing Gao, Wang Deng, Hongtao Jianghan Univ Sch Artificial Intelligence Wuhan Peoples R China Jianghan Univ Engn Res Ctr Intelligent Decis & Informat Proc Wuhan Peoples R China

Unsupervised keyphrase extraction (UKE) aims to detect out a set of keyphrases of a document without using any annotation signal for training the UKE model. Existing UKE models however either fail to solve the issue of low keyphrase relevance, or suffer from the problem of unsatisfactory keyphrase coverage. Besides, the current strong-performing neural based UKE methods unfortunately come with the cost of lower decoding efficiency. In this paper we propose a novel framework for UKE. We model the task as a fully generative process with an encoder-decoder framework, which unifies the prediction of both the present and absent keyphrases in an end-to-end manner. We accelerate the decoding by installing a non-autoregressive decoder, which yields all keyphrase extraction in parallel fast. To enhance the keyphrase relevance, we investigate a neural topic module that unsupervisingly learns the latent topic information, aiding the semantic understanding of the input document. To strengthen the keyphrase coverage, we present an anchor-region scheme for the extraction of present keyphrases, which first locates the anchor points of present phrases and then determines the final boundaries of keyphrases. The initial anchor phrases are organized into an anchor-aware graph, which is modeled by our proposed topic-guided graph attention network which captures the document contexts at global level. Over six benchmark datasets of UKE our system shows better experimental results than existing strong-performing baselines, becoming the new state-of-the-art UKE model. Further in-depth analyses show that our model can effectively resolve both the relevance and coverage of keyphrase extraction, meanwhile being faster on decoding.

关键词： natural language processing Keyphrase extraction Unsupervised learning Topic modeling graph attention network

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：