BERT is a widely used pre-trained model in naturallanguageprocessing tasks, including Aspect-Based sentiment classification. BERT is equipped with sufficient prior language knowledge in the enormous amount of pre-tr...
详细信息
ISBN:
(纸本)9781665463539
BERT is a widely used pre-trained model in naturallanguageprocessing tasks, including Aspect-Based sentiment classification. BERT is equipped with sufficient prior language knowledge in the enormous amount of pre-trained model parameters, for which the fine-tuning of BERT has become a critical issue. Previous works mainly focused on specialized downstream networks or additional knowledge to fine-tune the BERT to the sentiment classification tasks. In this paper, we design experiments to find the fine-tuning techniques that can be used by all models with BERT in the Aspect-Based Sentiment Classification tasks. Through these experiments, we verify different feature extraction, regularization, and continual learning methods, then we summarize 8 universally applicable conclusions to enhance the training and performance of the BERT model.
In the contemporary landscape of recruitment, the Applicant Tracking System (ATS) plays a pivotal role in automating the screening and shortlisting of candidates. However, the prevailing ATS encounters challenges such...
详细信息
In the contemporary landscape of recruitment, the Applicant Tracking System (ATS) plays a pivotal role in automating the screening and shortlisting of candidates. However, the prevailing ATS encounters challenges such as imprecise data extraction, erroneous keyword selection, and a lack of standardized criteria for comparison. As a result, many deserving applicants are turned away, highlighting the necessity for a more complex and human-centered strategy. In response to these limitations, our research introduces an innovative Resume Parsing and Ranking solution. Leveraging advanced naturallanguageprocessing techniques and machine learning algorithms, our system provides a customized experience for the automated screening process. The naive methods underscore the distinct advantages of our innovative approach, emphasizing the need to build a robust and accurate model for Resume Parsing and Ranking. Notably, it addresses discrepancies arising from diverse resume structures, ensuring a standardized and equitable evaluation of all applicants. The main contribution of our work lies in the development of a state-of-the-art Resume Parser that enhances efficiency, reduces bias, and optimizes candidate selection outcomes. Our proposed method integrates cutting-edge technologies to refine the existing ATS process, offering a tailored and precise approach to resume evaluation. The primary problem addressed is the lack of precision and standardization in thecurrent ATS, leading to suboptimal candidate shortlisting. Our solution tackles this by introducing a comprehensive framework that mitigates the impact of varied resume structures, thereby promoting fair and consistent candidate assessment. Through empirical validation, our obtained results showcase an accuracy of 96.2% in resume parsing, thereby significantly improving the efficiency of the candidate selection process.
Contrastive learning (CL) brought significant progress to various NLP tasks. Despite this progress, CL has not been applied to Arabic NLP to date. Nor is it clear how much benefits it could bring to particular classes...
详细信息
Text clustering is one of the central problems in text mining and information retrieval area. For the high dimensionality of feature space and the inherent data sparsity, performance of clustering algorithms will dram...
详细信息
ISBN:
(纸本)0780393619
Text clustering is one of the central problems in text mining and information retrieval area. For the high dimensionality of feature space and the inherent data sparsity, performance of clustering algorithms will dramatically decline. Two techniques are used to deal with this problem: feature extraction and feature selection. Feature selection methods have been successfully applied to text categorization but seldom applied to text clustering due to the unavailability of class label information. In this paper, four unsupervised feature selection methods, DF, TC, TVQ, and a new proposed method TV are introduced. Experiments are taken to show that feature selection methods can improves efficiency as well as accuracy of text clustering. Three clustering validity criterions are studied and used to evaluate clustering results.
We propose a method for generating paraphrases of English questions that retain the original intent but use a different surface form. Our model combines a careful choice of training objective with a principled informa...
详细信息
ISBN:
(纸本)9781954085527
We propose a method for generating paraphrases of English questions that retain the original intent but use a different surface form. Our model combines a careful choice of training objective with a principled information bottleneck, to induce a latent encoding space that disentangles meaning and form. We train an encoder-decoder model to reconstruct a question from a paraphrase with the same meaning and an exemplar with the same surface form, leading to separated encoding spaces. We use a Vector-Quantized Variational Autoencoder to represent the surface form as a set of discrete latent variables, allowing us to use a classifier to select a different surface form at test time. Crucially, our method does not require access to an external source of target exemplars. Extensive experiments and a human evaluation show that we are able to generate paraphrases with a better tradeoff between semantic preservation and syntactic novelty compared to previous methods.
Pre-trained language models (PLMs) have achieved great success in naturallanguageprocessing. Most of PLMs follow the default setting of architecture hyper-parameters (e.g., the hidden dimension is a quarter of the i...
详细信息
ISBN:
(纸本)9781954085527
Pre-trained language models (PLMs) have achieved great success in naturallanguageprocessing. Most of PLMs follow the default setting of architecture hyper-parameters (e.g., the hidden dimension is a quarter of the intermediate dimension in feed-forward sub-networks) in BERT (Devlin et al., 2019). Few studies have been conducted to explore the design of architecture hyper-parameters in BERT, especially for the more efficient PLMs with tiny sizes, which are essential for practical deployment on resource-constrained devices. In this paper, we adopt the one-shot Neural Architecture Search (NAS) to automatically search architecture hyper-parameters. Specifically, we carefully design the techniques of one-shot learning and the search space to provide an adaptive and efficient development way of tiny PLMs for various latency constraints. We name our method AutoTinyBERT(1) and evaluate its effectiveness on the GLUE and SQuAD benchmarks. The extensive experiments show that our method outperforms both the SOTA searchbased baseline (NAS-BERT) and the SOTA distillation-based methods (such as DistilBERT, TinyBERT, MiniLM and MobileBERT). In addition, based on the obtained architectures, we propose a more efficient development method that is even faster than the development of a single PLM.
The Japanese WordNet currently has 51,000 synsets with Japanese entries. In this paper, we discuss three methods of extending it: increasing the cover, linking it to examples in corpora and linking it to other resourc...
详细信息
In this paper, we address the task of language-independent, knowledge-lean and unsupervised compound splitting, which is an essential component for many naturallanguageprocessing tasks such as machine translation. P...
详细信息
naturallanguage generation (NLG) is a subfield of naturallanguageprocessing (NLP) that is often characterized as the study of automatically converting non-linguistic representations (e.g., from databases or other k...
详细信息
ISBN:
(数字)9783642155734
ISBN:
(纸本)9783642155727
naturallanguage generation (NLG) is a subfield of naturallanguageprocessing (NLP) that is often characterized as the study of automatically converting non-linguistic representations (e.g., from databases or other knowledge sources) into coherent naturallanguage text. In recent years the field has evolved substantially. Perhaps the most important new development is the current emphasis on data-oriented methods and empirical evaluation. Progress in related areas such as machine translation, dialogue system design and automatic text summarization and the resulting awareness of the importance of language generation, the increasing availability of suitable corpora in recent years, and the organization of shared tasks for NLG, where different teams of researchers develop and evaluate their algorithms on a shared, held out data set have had a considerable impact on the field, and this book offers the first comprehensive overview of recent empirically oriented NLG research.
Pre-trained language Models are widely used in many important real-world applications. However, recent studies show that these models can encode social biases from large pre-training corpora and even amplify biases in...
详细信息
ISBN:
(纸本)9798891760615
Pre-trained language Models are widely used in many important real-world applications. However, recent studies show that these models can encode social biases from large pre-training corpora and even amplify biases in downstream applications. To address this challenge, we propose (CoPT)-P-2, an efficient and effective debiaswhile-prompt tuning method for mitigating biases via counterfactual contrastive prompt tuning on downstream tasks. Our experiments conducted on three extrinsic bias benchmarks demonstrate the effectiveness of (CoPT)-P-2 on bias mitigation during the prompt tuning process and its adaptability to existing upstream debiased language models. These findings indicate the strength of (CoPT)-P-2 and provide promising avenues for further enhancement in bias mitigation on downstream tasks.
暂无评论