Dialogue act (DA) recognition is crucial in many naturallanguageprocessing tasks. In dialogues, speakers aim to express their acts, which may be conveyed implicitly or explicitly. Therefore, DA serves as a semantic ...
详细信息
ISBN:
(数字)9798350359312
ISBN:
(纸本)9798350359329
Dialogue act (DA) recognition is crucial in many naturallanguageprocessing tasks. In dialogues, speakers aim to express their acts, which may be conveyed implicitly or explicitly. Therefore, DA serves as a semantic label for utterances and is essential for understanding the speaker’s action. With the widespread application of pre-trained language models in text recognition tasks, methods based on pre-trained language models can learn feature information from texts. However, most current methods do not consider the structural features of previous and subsequent dialogue utterances. The T5 model has recently performed excellently in various naturallanguageprocessing tasks. This paper proposes a model that uses the T5 language model to recognize DA. Moreover, considering the unique characteristics of dialogue text, we add a self-supervised pre-training task before fine-tuning. By combining dialogue text and DA, using continuous multiple utterances as input to learn the structural features of dialogue text and assist in fine-tuning tasks, we utilize a generative approach for DA recognition during the fine-tuning phase. Experimental results show that on the DailyDialog dataset, the proposed model achieved an F1 score of 84.1%, which is 2.8 percentage points higher than the T5 model, thus validating the superiority of the proposed model.
Learning Chinese word embeddings is important in many tasks of Chinese language information processing, such as entity linking, entity extraction, and knowledge graph. A Chinese word consists of Chinese characters, wh...
详细信息
ISBN:
(纸本)9781450392365
Learning Chinese word embeddings is important in many tasks of Chinese language information processing, such as entity linking, entity extraction, and knowledge graph. A Chinese word consists of Chinese characters, which can be decomposed into sub-characters (radical, component, stroke, etc). Similar to roots in English words, sub-characters also indicate the origins and basic semantics of Chinese characters. So, many researches follow the approaches designed for learning embeddings of English words to improve Chinese word embeddings. However, some Chinese characters sharing the same sub-characters have different meanings. Furthermore, with more cultural interaction and the popularization of the Internet and web, many neologisms, such as transliterated loanwords and network terms, are emerging, which are only close to the pronunciation of their characters, but far from their semantics. Here, a tripartite weighted graph is proposed to model the semantic relationship among words, characters, and sub-characters, in which the semantic relationship is evaluated according to the Chinese linguistic information. So, the semantic relevance hidden in lower components (sub-characters, characters) can be used to further distinguish the semantics of corresponding higher components (characters, words). Then, the tripartite weighted graph is fed into our Chineseword embedding model insideCC to reveal the semantic relationship among different language components, and learn the embeddings of words. Extensive experimental results on multiple corpora and datasets verify that our proposed methods outperform the state-of-the-art counterparts by a significant margin.
Word sense disambiguation is a very important task in naturallanguageprocessing, and it is also a basic work in this field. There are many polysemous words in Chinese vocabulary. Using the word sense disambiguation ...
详细信息
Syllabification is a crucial task in naturallanguageprocessing, and syllables also play a significant role as modeling units in speech processing. While deep learning methods have shown remarkable progress in syllab...
详细信息
ISBN:
(数字)9798350359312
ISBN:
(纸本)9798350359329
Syllabification is a crucial task in naturallanguageprocessing, and syllables also play a significant role as modeling units in speech processing. While deep learning methods have shown remarkable progress in syllabification, they face challenges in low-resource languages where ready-made segmentation datasets or rules are lacking. Large language models (LLMs) are mostly unsuitable for these low-resource languages as well. To address these challenges, this paper proposes an unsupervised syllabification approach that incorporates logical reasoning into the reinforcement learning training process, achieving knowledge-guided syllabification. By introducing logical reasoning knowledge and modeling the interaction between the Agent and the knowledge base (KB), the model gains a better understanding of language structures and patterns. The study primarily focuses on low-resource Lao language, with experiments conducted on a publicly available English dataset to validate the effectiveness of the proposed method.
Sentiment analysis is nothing but an information retrieval system based on naturallanguageprocessing (NLP). It is a machine learning algorithm. The rudimentary prototype of sentiment analysis comes under the classif...
详细信息
ISBN:
(纸本)9789811965814;9789811965807
Sentiment analysis is nothing but an information retrieval system based on naturallanguageprocessing (NLP). It is a machine learning algorithm. The rudimentary prototype of sentiment analysis comes under the classifier problem where the negative, positive, and neutral are the classes we would be expecting as an output, given text (tweets in our case) as an input. Feature selection methods (information gain and chi square) and feature extraction will be performed to get the output sentiment. After studying three classifiers, support vector machine (SVM), K-nearest neighbour (KNN), and Naive Bayes, the latter algorithm, i.e. Naive Bayes being most accurate among them would be considered for implementation. As mentioned, we would be focusing on major three output sentiment classes: positive, negative, and neutral. Further, we are getting into a detailed analysis of tweets reflecting a depressive state of mind due to sadness. Though our ultimate goal is to predict the mental health of a user by observing the consistency and frequency of tweets along with its sentiment analysis, for now, we are just dealing with one tweet per user at a time.
Model quantification uses low bit-width values to represent the weight matrices of existing models to be quantized, which is a promising approach to reduce both storage and computational overheads of deploying highly ...
Joint entity and relation extraction is an essential task in naturallanguageprocessing and knowledge graph construction. Existing approaches usually decompose the joint extraction task into several basic modules or ...
详细信息
ISBN:
(纸本)9781577358763
Joint entity and relation extraction is an essential task in naturallanguageprocessing and knowledge graph construction. Existing approaches usually decompose the joint extraction task into several basic modules or processing steps to make it easy to conduct. However, such a paradigm ignores the fact that the three elements of a triple are interdependent and indivisible. Therefore, previous joint methods suffer from the problems of cascading errors and redundant information. To address these issues, in this paper, we propose a novel joint entity and relation extraction model, named OneRel, which casts joint extraction as a fine-grained triple classification problem. Specifically, our model consists of a scoring-based classifier and a relation-specific horns tagging strategy. The former evaluates whether a token pair and a relation belong to a factual triple. The latter ensures a simple but effective decoding process. Extensive experimental results on two widely used datasets demonstrate that the proposed method performs better than the state-of-the-art baselines, and delivers consistent performance gain on complex scenarios of various overlapping patterns and multiple triples.
methods such as knowledge-enabled language representation model (K-BERT) that help train models using external information, such as knowledge graphs, have recently been proposed in the field of naturallanguage proces...
详细信息
With the advent of the SG era, high-speed and secure network access services have become a common pursuit. The QUIC (Quick UDP Internet Connection) protocol proposed by Google has been studied by many scholars due to ...
详细信息
ISBN:
(纸本)9781665406079
With the advent of the SG era, high-speed and secure network access services have become a common pursuit. The QUIC (Quick UDP Internet Connection) protocol proposed by Google has been studied by many scholars due to its high speed, robustness, and low latency. However, the research on the security of the QUIC protocol by domestic and foreign scholars is insufficient. Therefore, based on the self-similarity of QUIC network traffic, combined with traffic characteristics and signal processingmethods, a QUIC-based network traffic anomaly detection model is proposed in this paper. The model decomposes and reconstructs the collected QUIC network traffic data through the empirical Mode Decomposition (EMD) method. In order to judge the occurrence of abnormality, this paper also intercepts overlapping traffic segments through sliding windows to calculate Hurst parameters and analyzes the obtained parameters to check abnormal traffic. The simulation results show that in the network environment based on the QUIC protocol, the Hurst parameter after being attacked fluctuates violently and exceeds the normal range. It also shows that the anomaly detection of QUIC network traffic can use the EMD method.
Distantly supervised relation extraction (DSRE) generates large-scale annotated data by aligning unstructured text with knowledge bases. However, automatic construction methods cause a substantial number of incorrect ...
详细信息
ISBN:
(数字)9789819723904
ISBN:
(纸本)9789819723898;9789819723904
Distantly supervised relation extraction (DSRE) generates large-scale annotated data by aligning unstructured text with knowledge bases. However, automatic construction methods cause a substantial number of incorrect annotations, thereby introducing noise into the training process. Most sentence-level relation extraction methods rely on filters to remove noise instances, meanwhile, they ignore some useful information in negative instances. To effectively reduce noise interference, we propose a Multi-teacher Knowledge Distillation framework for Relation Extraction (MKDRE) to extract semantic relations from noisy data based on both global information and local information. MKDRE addresses two main problems: the deviation in knowledge propagation of a single teacher and the limitation of traditional distillation temperature on information utilization. Specifically, we utilize flexible temperature regulation (FTR) to adjust the temperature assigned to each training instance, so as to dynamically capture local relations between instances. Furthermore, we introduce information entropy of hidden layers to gain stable temperature calculations. Finally, we propose multi-view knowledge distillation (MVKD) to express global relations among teachers from various perspectives to gain more reliable knowledge. The experimental results on NYT19-1.0 and NYT19-2.0 datasets show that our proposed MKDRE significantly outperforms previous methods in sentence-level relation extraction.
暂无评论