This paper describes our system for the submission to the TextGraphs 2022 shared task at COLING 2022: naturallanguage Premise Selection (NLPS) from mathematical texts. The task of NLPS regards selecting mathematical ...
详细信息
Metric-based learning is a well-known family of methods for few-shot learning, especially in computer vision. Recently, they have been used in many naturallanguageprocessing applications but not for slot tagging. In...
详细信息
ISBN:
(纸本)9781954085022
Metric-based learning is a well-known family of methods for few-shot learning, especially in computer vision. Recently, they have been used in many naturallanguageprocessing applications but not for slot tagging. In this paper, we explore metric-based learning methods in the slot tagging task and propose a novel metric-based learning architecture - Attentive Relational Network. Our proposed method extends relation networks, making them more suitable for naturallanguageprocessing applications in general, by leveraging pretrained contextual embeddings such as ELMO and BERT and by using attention mechanism. The results on SNIPS data show that our proposed method outperforms other state of the art metric-based learning methods.
We study the problem of integrating cognitive languageprocessing signals (e.g., eyetracking or EEG data) into pre-trained models like BERT. Existing methods typically fine-tune pre-trained models on cognitive data, i...
详细信息
Convolutional neural networks (CNNs) have gained significant popularity in the field of hyperspectral image (HSI) classification due to their ability to capture detailed features. However, the existed CNN-based HSI cl...
详细信息
ISBN:
(数字)9798350349115
ISBN:
(纸本)9798350349122
Convolutional neural networks (CNNs) have gained significant popularity in the field of hyperspectral image (HSI) classification due to their ability to capture detailed features. However, the existed CNN-based HSI classification methods primarily rely on patch-based learning approaches. These methods do not fully utilize global information and also require high computational resources. To address these issues, we propose a novel image-based global learning framework for HSI classification. Within this framework, we introduce a Global-Multiscale Channel Convolutional Network that effectively exploits the global and multiscale information from HSI. In addition, we propose a novel spatial attention module. The module can mitigate the decline in performance that occurs with an increase of the input patch size by capturing the homogeneous pixels in the input patch. Our experimental results, conducted on two real hyperspectral datasets, demonstrate that our method outperforms other related methods in terms of both efficiency and accuracy for HSI land cover classification.
The advent of pre-trained language models (PLMs) has revolutionized the field of naturallanguageprocessing (NLP), enabling models to leverage vast amounts of language and world knowledge for various downstream tasks...
详细信息
ISBN:
(数字)9798350349115
ISBN:
(纸本)9798350349122
The advent of pre-trained language models (PLMs) has revolutionized the field of naturallanguageprocessing (NLP), enabling models to leverage vast amounts of language and world knowledge for various downstream tasks. However, the conventional fine-tuning of these models requires updating full parameters, which is computationally intensive and impractical in resource-constrained settings. Parameter-efficient tuning methods have been developed to mitigate this. However, these methods may need to be improved in terms of inference latency, optimization difficulty, and input sequence length *** this paper, we introduce a new reparameterization-based parameter-efficient tuning method, Additive Delta Tuning (ADT). ADT only fine-tunes part of the row and column parameters in the PLM weight matrix. This approach significantly reduces the computational cost compared to existing methods, such as Low-Rank Adaptation (LoRA), while maintaining competitive performance. We conducted extensive experiments across multiple naturallanguage understanding (NLU) tasks, including text classification, text entailment, named entity recognition, etc., to evaluate the effectiveness of ADT. Our experimental results demonstrate that ADT achieves comparable or better performance than traditional fine-tuning methods, requiring less than 0.3% parameter updates. This shows that ADT is an effective method for parameter-efficient tuning. Our method achieves comparable results to LoRA on multiple NLU tasks, with the added benefit of requiring less computational cost.
naturallanguageprocessing as the current science and technology innovation and the main direction of artificial intelligence research and needs to be communication between man and machine as a fusion algorithm of co...
详细信息
In recent years, depression, as a serious mental illness, has received widespread attention from various sectors of society. How to identify depressive emotions in a timely manner and detect depression has become an u...
详细信息
We study algorithms for approximating pairwise similarity matrices that arise in naturallanguageprocessing. Generally, computing a similarity matrix for n data points requires Omega(n(2)) similarity computations. Th...
详细信息
ISBN:
(纸本)9781577358763
We study algorithms for approximating pairwise similarity matrices that arise in naturallanguageprocessing. Generally, computing a similarity matrix for n data points requires Omega(n(2)) similarity computations. This quadratic scaling is a significant bottleneck, especially when similarities are computed via expensive functions, e.g., via transformer models. Approximation methods reduce this quadratic complexity, often by using a small subset of exactly computed similarities to approximate the remainder of the complete pairwise similarity matrix. Significant work focuses on the efficient approximation of positive semidefinite (PSD) similarity matrices, which arise e.g., in kernel methods. However, much less is understood about indefinite (non-PSD) similarity matrices, which often arise in NLP. Motivated by the observation that many of these matrices are still somewhat close to PSD, we introduce a generalization of the popular Nystrom method to the indefinite setting. Our algorithm can be applied to any similarity matrix and runs in sublinear time in the size of the matrix, producing a rank-8 approximation with just O(ns) similarity computations. We show that our method, along with a simple variant of CUR decomposition, performs very well in approximating a variety of similarity matrices arising in NLP tasks. We demonstrate high accuracy of the approximated similarity matrices in tasks of document classification, sentence similarity, and cross-document coreference.
In the interest of interpreting neural NLI models and their reasoning strategies, we carry out a systematic probing study which investigates whether these models capture the crucial semantic features central to natura...
详细信息
naturallanguageprocessing has been a challenging area for researchers in recent times. A lot of research works are conducted to improve the performance of naturallanguageprocessing that have helped many applicatio...
详细信息
ISBN:
(纸本)9781728158754
naturallanguageprocessing has been a challenging area for researchers in recent times. A lot of research works are conducted to improve the performance of naturallanguageprocessing that have helped many applications to be made available for our day to day life. Most of the developments have happened only for a few western languages. Researchers have also published some quality works related to Indian languages in recent times. However, the research works conducted for most of the other languages spoken in India, the country with the second largest population in the world, are at a very primitive stage. This paper aims to review the advances in this area for one of the Low Resource Indian languages -"Assamese", the official language of Assam, the gateway to North East India. It is also spoken in many other states of North Eastern part of India. In this work, we present a highlight of research works related to various methods that are applied to Assamese Text processing. It is observed that certain language characteristics of Assamese are best applicable to certain popular methods used for naturallanguageprocessing. The reason behind this relook is to overview and report the present state and future possibilities of research in text processing in Assamese. This paper should serve as a good beginning for anyone interested to carry forward the computational works for Assamese. We conclude the paper with a brief discussion on the scope of Assamese Text processing in the future.
暂无评论