检索结果-内蒙古大学图书馆

hmBERT: Historical Multilingual Language Models for Named Entity Recognition

学校读者我要写书评

暂无评论

hmBERT: Historical Multilingual Language Models for Named En...

2022 Conference and labs of the Evaluation Forum, CLEF 2022

作者： Schweter, Stefan März, Luisa Schmid, Katharina Çano, Erion Bayerische Staatsbibliothek München Digital Library/ Munich Digitization Center Munich Germany Digital Philology Research Group Data Mining and Machine Learning University of Vienna Austria Natural Language Processing Expert Center Data:Lab Volkswagen AG Munich Germany

Compared to standard Named Entity Recognition (NER), identifying persons, locations, and organizations in historical texts constitutes a big challenge. To obtain machine-readable corpora, the historical text is usually scanned and Optical Character Recognition (OCR) needs to be performed. As a result, the historical corpora contain errors. Also, entities like location or organization can change over time, which poses another challenge. Overall, historical texts come with several peculiarities that differ greatly from modern texts and large labeled corpora for training a neural tagger are hardly available for this domain. In this work, we tackle NER for historical German, English, French, Swedish, and Finnish by training large historical language models. We circumvent the need for large amounts of labeled data by using unlabeled data for pretraining a language model. We propose hmBert, a historical multilingual BERT-based language model, and release the model in several versions of different sizes. Furthermore, we evaluate the capability of hmBert by solving downstream NER as part of this year’s HIPE-2022 shared task and provide detailed analysis and insights. For the Multilingual Classical Commentary coarse-grained NER challenge, our tagger HISTeria outperforms the other teams’ models for two out of three languages. © 2022 Copyright for this paper by its authors.

关键词： Optical character recognition

Effective resampling approach for skewed distribution on imbalanced data set

学校读者我要写书评

暂无评论

IAENG International Journal of Computer Science

IAENG International Journal of Computer Science 2020年第2期47卷 234-249页

作者： Nwe, Mar Mar Lynn, Khin Thidar Data Mining and Machine Learning Lab University of Computer Studies Mandalay Myanmar Faculty of Information Science Department University of Computer Studies Mandalay Myanmar

Accurate classification of unknown input data for imbalanced data sets is difficult, because the predictions of learning classifiers tend to be biased towards the majority class and ignore the minority class. Moreover, the class distribution of imbalanced data has a significant impact on the misclassification rate of the learning classifier. So, this paper introduces an effective data pre-processing approach to improve the efficiency of imbalanced data classification, focusing on the skewed distribution of data points in the imbalanced data set. This proposed approach involves over-sampling and under-sampling techniques based on k-means clustering to overcome the problems associated with imbalanced learning of small disjuncts and small sample size. And, Tomek Link-based under-sampling method is also incorporated into the proposed cluster-based resampling methods to solve the class overlapping problem by eliminating the majority samples in overlapping regions. Experiments are performed on the 25 standard imbalanced data sets by applying four learning classifiers, and validated with the three popular metrics (i.e. Area Under the Curve (AUC), Geometric Mean (G-mean) and Balanced Accuracy (BA)). Specially, we show that the proposed approach has outperformed the other state-of-the-art resampling methods using performance metrics, probabilistic estimation, statistical analysis and multicriteria decision-making methodology (MCDM). © 2020, International Association of Engineers.

关键词： Classification (of information)

Handling the Concept Drifts Based on Ensemble learning with Adaptive Windows

学校读者我要写书评

暂无评论

IAENG International Journal of Computer Science

IAENG International Journal of Computer Science 2021年第3期48卷 1-16页

作者： Myint, Tin Mar Lynn, Khin Thidar Candidate at Data Mining and Machine Learning Lab University of Computer Studies Mandalay Myanmar Professor at Faculty of Information Science University of Computer Studies Mandalay Myanmar

Continuous learning from streaming data is one of the contemporary most challenging topics. learning algorithms not only need to handle fast-moving big data but should also be able to adapt to future evolving changes. The evolving structure of data streams is the phenomenon referred to as concept drift. Two types of concept drift handling based on ensemble approaches are online ensemble and chunk-based ensemble (window-based ensemble). The main disadvantage of the chunk-based ensembles is the difficulty of tuning the block size to provide a trade-off between fast reactions to drifts. In this paper, we propose Accuracy Updated Ensemble-2 (AUE2) based on the adaptive windowing approach (denoted as A-AUE2) by using Brier Skill Score as sudden and gradual changes detector. Moreover, K-Nearest Neighbors (KNN) based noise filtering method is applied to eliminate the noisy samples from each of the adaptive windows to improve the effectiveness of the ensemble learning approach. This paper's proposed approach has been evaluated on six artificial data sets and four real benchmark data sets by using two base learners. Specially, we illustrate that the proposed approach has outperformed the other state-of-the-art drift detection and handling approaches by using performance metrics and statistical analysis. © 2021. All Rights Reserved.

关键词： learning algorithms

KnowMAN: Weakly supervised multinomial adversarial networks

学校读者我要写书评

暂无评论

arXiv 2021年

作者： März, Luisa Asgari, Ehsaneddin Braune, Fabienne Zimmermann, Franziska Roth, Benjamin Digital Philology Research Group Data Mining and Machine Learning University of Vienna Austria Nlp Expert Center Data:Lab Volkswagen Ag Munich Germany

The absence of labeled data for training neural models is often addressed by leveraging knowledge about the specific task, resulting in heuristic but noisy labels. The knowledge is captured in labeling functions, which detect certain regularities or patterns in the training samples and annotate corresponding labels for training. This process of weakly supervised training may result in an over-reliance on the signals captured by the labeling functions and hinder models to exploit other signals or to generalize well. We propose KnowMAN, an adversarial scheme that enables to control influence of signals associated with specific labeling functions. KnowMAN forces the network to learn representations that are invariant to those signals and to pick up other signals that are more generally associated with an output label. KnowMAN strongly improves results compared to direct weakly supervised learning with a pre-trained transformer language model and a feature-based baseline. © 2021, CC BY.

关键词： Computational linguistics

Local and Global Information Preserved Network Embedding

学校读者我要写书评

暂无评论

Local and Global Information Preserved Network Embedding

International Conference on Advances in Social Network Analysis and mining, ASONAM

作者： Yao Ma Suhang Wang Jiliang Tang Data Science and Engineering Lab Michigan State University East Lansing MI USA Data Mining and Machine Learning Lab Arizona State University Tempe AZ USA

ISBN: (纸本)9781538660522

Networks such as social networks, airplane networks, and citation networks are ubiquitous. To apply advanced machine learning algorithms to network data, low-dimensional and continuous representations are desired. To achieve this goal, many network embedding methods have been proposed recently. The majority of existing methods facilitate the local information i.e. local connections between nodes, to learn the representations, while neglecting global information (or node status), which has been proven to boost numerous network mining tasks such as link prediction and social recommendation. In this paper, we study the problem of preserving local and global information for network embedding. In particular, we introduce an approach to capture global information and propose a network embedding framework LOG, which can coherently model LOcal and Global information. Experiments demonstrate the effectiveness of the proposed framework.

关键词： Task analysis Social network services Prediction algorithms Blogs Mathematical model Training Linear programming

HmBERT: Historical Multilingual Language Models for Named Entity Recognition

学校读者我要写书评

暂无评论

arXiv 2022年

Compared to standard Named Entity Recognition (NER), identifying persons, locations, and organizations in historical texts constitutes a big challenge. To obtain machine-readable corpora, the historical text is usually scanned and Optical Character Recognition (OCR) needs to be performed. As a result, the historical corpora contain errors. Also, entities like location or organization can change over time, which poses another challenge. Overall, historical texts come with several peculiarities that differ greatly from modern texts and large labeled corpora for training a neural tagger are hardly available for this domain. In this work, we tackle NER for historical German, English, French, Swedish, and Finnish by training large historical language models. We circumvent the need for large amounts of labeled data by using unlabeled data for pretraining a language model. We propose hmBert, a historical multilingual BERT-based language model, and release the model in several versions of different sizes. Furthermore, we evaluate the capability of hmBert by solving downstream NER as part of this year's HIPE-2022 shared task and provide detailed analysis and insights. For the Multilingual Classical Commentary coarse-grained NER challenge, our tagger HISTeria outperforms the other teams' models for two out of three languages. Copyright © 2022, The Authors. All rights reserved.

关键词： Optical character recognition

Efficient image representation for object recognition via pivots selection

学校读者我要写书评

暂无评论

Frontiers of Computer Science 2014年

作者： Xie, Bojun Liu, Yi Zhang, Hui Yu, Jian Beijing Key Lab of Traffic Data Analysis and Mining School of Computer and Information Technology Beijing Jiaotong University Beijing100044 China Key Lab of Machine Learning and Computational Intelligence College of Mathematics and Computer Science Hebei University Baoding071000 China

Patch-level features are essential for achieving good performance in computer vision tasks. Besides well-known pre-defined patch-level descriptors such as scaleinvariant feature transform (SIFT) and histogram of oriented gradient (HOG), the kernel descriptor (KD) method [1] offers a new way to "grow-up" features from a match-kernel defined over image patch pairs using kernel principal component analysis (KPCA) and yields impressive *** this paper, we present efficient kernel descriptor (EKD) and efficient hierarchical kernel descriptor (EHKD), which are built upon incomplete Cholesky decomposition. EKD automatically selects a small number of pivot features for generating patch-level features to achieve better computational efficiency. EHKD recursively applies EKD to form image-level features layer-by-layer. Perhaps due to parsimony, we find surprisingly that the EKD and EHKD approaches achieved competitive results on several public datasets compared with other state-of-the-art methods, at an improved efficiency over KD. © 2014 Higher Education Press and Springer-Verlag Berlin Heidelberg

关键词： Object recognition

Mixing Histopathology Prototypes into Robust Slide-Level Representations for Cancer Subtyping

学校读者我要写书评

暂无评论

arXiv 2023年

作者： Butke, Joshua Hashimoto, Noriaki Takeuchi, Ichiro Miyoshi, Hiroaki Ohshima, Koichi Sakuma, Jun Machine Learning and Data Mining Lab University of Tsukuba Japan RIKEN Center for Advanced Intelligence Project Japan Department of Mechanical Systems Engineering Nagoya University Japan Department of Pathology Kurume University Japan Department of Computer Science Tokyo Institute of Technology Japan

Whole-slide image analysis via the means of computational pathology often relies on processing tessellated gigapixel images with only slide-level labels available. Applying multiple instance learning-based methods or transformer models is computationally expensive as, for each image, all instances have to be processed simultaneously. The MLP-Mixer is an under-explored alternative model to common vision transformers, especially for large-scale datasets. Due to the lack of a self-attention mechanism, they have linear computational complexity to the number of input patches but achieve comparable performance on natural image datasets. We propose a combination of feature embedding and clustering to preprocess the full whole-slide image into a reduced prototype representation which can then serve as input to a suitable MLP-Mixer architecture. Our experiments on two public benchmarks and one inhouse malignant lymphoma dataset show comparable performance to current state-of-the-art methods, while achieving lower training costs in terms of computational time and memory load. Code is publicly available at https://***/butkej/ProtoMixer. Copyright © 2023, The Authors. All rights reserved.

关键词： Mixers (machinery)

data centric domain adaptation for historical text with OCR errors

学校读者我要写书评

暂无评论

arXiv 2021年

作者： März, Luisa Schweter, Stefan Poerner, Nina Roth, Benjamin Schütze, Hinrich Center for Information and Language Processing Ludwig Maximilian University Munich Germany Digital Philology Research Group Data Mining and Machine Learning University of Vienna Austria Bayerische Staatsbibliothek München Digital Library/Munich Digitization Center Munich Germany NLP Expert Center Data:Lab Volkswagen AG Munich Germany

We propose new methods for in-domain and cross-domain Named Entity Recognition (NER) on historical data for Dutch and French. For the cross-domain case, we address domain shift by integrating unsupervised in-domain data via contextualized string embeddings;and OCR errors by injecting synthetic OCR errors into the source domain and address data centric domain adaptation. We propose a general approach to imitate OCR errors in arbitrary input data. Our cross-domain as well as our in-domain results outperform several strong baselines and establish state-of-the-art results. We publish preprocessed versions of the French and Dutch Europeana NER corpora. © 2021, CC BY.

关键词： Errors