检索结果-内蒙古大学图书馆

St-Keys: Self-Supervised Transformer for Keyword Spotting in Historical Handwritten Documents

学校读者我要写书评

暂无评论

SSRN

SSRN 2023年

作者： Jemni, Sana Khamekhem Ammar, Sourour Souibgui, Mohamed Ali Kessentini, Yousri Cheddad, Abbas Digital Research Center of Sfax B.P. 275 Sakiet Ezzit Sfax3021 Tunisia SM@RTS: Laboratory of Signals systeMs aRtificial Intelligence and neTworkS MIR@CL: Multimedia InfoRmation systems and Advanced Computing Laboratory Computer Vision Center Computer Science Department Universitat Aut`onoma de Barcelona Spain Department of Computer Science Blekinge Institute of Technology Karlskrona Sweden

Keyword spotting (KWS) in historical documents is an important tool for the initial exploration of digitized collections. Nowadays, the most efficient KWS methods are relying on machine learning techniques that require a large amount of annotated training data. However, in the case of historical manuscripts, there is a lack of annotated corpus for training. To handle the data scarcity issue, we investigate the merits of the self-supervised learning to extract useful representations of the input data without relying on human annotations and then using these representations in the downstream task. We propose ST-KeyS, a masked auto-encoder model based on vision transformers where the pretraining stage is based on the mask-and-predict paradigm, without the need of labeled data. In the fine-tuning stage, the pre-trained encoder is integrated into a siamese neural network model that is fine-tuned to improve feature embedding from the input images. We further improve the image representation using pyramidal histogram of characters (PHOC) embedding to create and exploit an intermediate representation of images based on text attributes. In an exhaustive experimental evaluationon three widely used benchmark datasets (Botany, Alvermann Konzilsprotokolle and George Washington), the proposed approach outperforms state-of-the-art methods trained on the same datasets. © 2023, The Authors. All rights reserved.

关键词： Embeddings

InfoDiffusion: Information Entropy Aware Diffusion Process for Non-Autoregressive Text Generation

学校读者我要写书评

暂无评论

arXiv 2023年

作者： Wang, Renzhi Li, Jing Li, Piji College of Computer Science and Technology Nanjing University of Aeronautics and Astronautics China MIIT Key Laboratory of Pattern Analysis and Machine Intelligence Nanjing China Department of Computing The Hong Kong Polytechnic University China

Diffusion models have garnered considerable interest in the field of text generation. Several studies have explored text diffusion models with different structures and applied them to various tasks, including named entity recognition and summarization. However, there exists a notable disparity between the "easy-first" text generation process of current diffusion models and the "keyword-first" natural text generation process of humans, which has received limited attention. To bridge this gap, we propose InfoDiffusion, a non-autoregressive text diffusion model. Our approach introduces a "keyinfo-first" generation strategy and incorporates a noise schedule based on the amount of text information. In addition, InfoDiffusion combines self-conditioning with a newly proposed partially noising model structure. Experimental results show that InfoDiffusion outperforms the baseline model in terms of generation quality and diversity, as well as exhibiting higher sampling efficiency.1 Copyright © 2023, The Authors. All rights reserved.

关键词： Character recognition

Neural P3M: a long-range interaction modeling enhancer for geometric GNNs 24

学校读者我要写书评

暂无评论

Neural P3M: a long-range interaction modeling enhancer for g...

Proceedings of the 38th International Conference on Neural Information Processing Systems

作者： Yusong Wang Chaoran Cheng Shaoning Li Yuxuan Ren Bin Shao Ge Liu Pheng-Ann Heng Nanning Zheng National Key Laboratory of Human-Machine Hybrid Augmented Intelligence National Engineering Research Center for Visual Information and Applications and Institute of Artificial Intelligence and Robotics Xi'an Jiaotong University University of Illinois Urbana-Champaign Department of Computer Science and Engineering The Chinese University of Hong Kong University of Science and Technology of China Microsoft Research AI4Science

ISBN: (纸本)9798331314385

Geometric graph neural networks (GNNs) have emerged as powerful tools for modeling molecular geometry. However, they encounter limitations in effectively capturing long-range interactions in large molecular systems due to the localization assumption of GNN. To address this challenge, we introduce Neural P3M, a versatile enhancer of geometric GNNs to expand the scope of their capabilities by incorporating mesh points alongside atoms and reimaging traditional mathematical operations in a trainable manner. Neural P3M exhibits flexibility across a wide range of molecular systems and demonstrates remarkable accuracy in predicting energies and forces, outperforming on benchmarks such as the MD22 dataset. It also achieves an average improvement of 22% on the OE62 dataset while integrating with various architectures. Codes are available at https://***/OnlyLoveKFC/Neural_P3M.

关键词：

An Explainable Coarse-to-Fine Survival Analysis Method on Multi-Center Whole Slide Images

学校读者我要写书评

暂无评论

IEEE Transactions on Artificial Intelligence

IEEE Transactions on Artificial intelligence 2024年第3期5卷 1316-1327页

作者： Wang, Han Jiang, Dan Zhang, Haixian Wang, Yujue Yang, Li Kerr, David J. Zhang, Yi Sichuan University-University of Oxford Huaxi Joint Centre for Gastrointestinal Cancer West China Hospital Sichuan University Chengdu610041 China Machine Intelligence Laboratory College of Computer Science Sichuan University Chengdu610065 China Beijing StoneWise Technology Company Ltd. Beijing100080 China Department of Pathology West China Hospital Sichuan University Chengdu610041 China West China School of Basic Medical Sciences and Forensic Medicine Sichuan University Chengdu610041 China Department of Gastroenterology and Hepatology West China Hospital Sichuan University Chengdu610041 China Radcliffe Department of Medicine University of Oxford OxfordOX3 9DU United Kingdom

Survival models based on whole slide images (WSIs) arewidely used in precision medicine to treat cancer patients better. Most previous studies attempted to address the challenge ofWSIs' gigapixel resolutions to survival models, but they failed in terms of computational efficiency and interpretability ofmodels. This study proposes a coarse-to-fine survival model called WSISur based on graph neural networks, which not only solves the above two problems but also achieves the best survival prediction performance. To solve the issue of computational efficiency, coarse WSI graphs are first constructed on low-resolution images in WSIs, and then fine WSI graphs are built with high-resolution images on the basis of coarse WSI graphs. Subsequent survival analysis is performed on the constructed WSI graphs. To solve the issue of interpretability of the model,WSIs' regions most relevant to patients' lifetimes are identified by gradient-weighted class activation mapping. Nevertheless, due to the imbalance of data labels, there is a problem of probability failure in end-to-end mini-batch training. To this end, a survival event sampling strategy is proposed to balance data labels. In addition, a series of experiments are carried out on multicenter datasets to evaluate the proposed *** results show that WSISur achieves the state-of-the-art result compared with other existing related methods, and has stronger domain invariance. Results also demonstrate the interpretability of WSISur. Impact Statement-Survival models based on WSI play an important role in precision medicine. Existing WSI based survival models have lowcomputational efficiency and lack of interpretability. A novel survival model we propose in this study overcomes these limitations. The proposed survival model is not only computationally efficient but also interpretable. Moreover, the proposed survival model also shows a performance improvement of nearly 23%~29% over existing methods on multi-center datas

关键词： Graphic methods

TransXNet: Learning Both Global and Local Dynamics with a Dual Dynamic Token Mixer for Visual Recognition

学校读者我要写书评

暂无评论

arXiv 2023年

作者： Lou, Meng Zhou, Hong-Yu Yang, Sibei Yu, Yizhou The Artificial Intelligence Laboratory Deepwise Healthcare Beijing China The Department of Computer Science The University of Hong Kong Hong Kong The Shanghai Engineering Research Center of Intelligent Vision and Imaging ShanghaiTech University Shanghai China

Recent studies have integrated convolution into transformers to introduce inductive bias and improve generalization performance. However, the static nature of conventional convolution prevents it from dynamically adapting to input variations, resulting in a representation discrepancy between convolution and self-attention as self-attention calculates attention matrices dynamically. Furthermore, when stacking token mixers that consist of convolution and self-attention to form a deep network, the static nature of convolution hinders the fusion of features previously generated by self-attention into convolution kernels. These two limitations result in a suboptimal representation capacity of the constructed networks. To find a solution, we propose a lightweight Dual Dynamic Token Mixer (D-Mixer) that aggregates global information and local details in an input-dependent way. D-Mixer works by applying an efficient global attention module and an input-dependent depthwise convolution separately on evenly split feature segments, endowing the network with strong inductive bias and an enlarged effective receptive field. We use D-Mixer as the basic building block to design TransXNet, a novel hybrid CNN-Transformer vision backbone network that delivers compelling performance. In the ImageNet-1K image classification task, TransXNet-T surpasses Swin-T by 0.3% in top-1 accuracy while requiring less than half of the computational cost. Furthermore, TransXNet-S and TransXNet-B exhibit excellent model scalability, achieving top-1 accuracy of 83.8% and 84.6% respectively, with reasonable computational costs. Additionally, our proposed network architecture demonstrates strong generalization capabilities in various dense prediction tasks, outperforming other state-of-the-art networks while having lower computational costs. Our code will be available at https://***/LMMMEng/TransXNet. © 2023, CC BY.

关键词： Convolution

EFFICIENT ONLINE LABEL CONSISTENT HASHING FOR LARGE-SCALE CROSS-MODAL RETRIEVAL

学校读者我要写书评

暂无评论

EFFICIENT ONLINE LABEL CONSISTENT HASHING FOR LARGE-SCALE CR...

2021 IEEE International Conference on Multimedia and Expo, ICME 2021

作者： Yi, Jinhan Liu, Xin Cheung, Yiu-Ming Xu, Xing Fan, Wentao He, Yi Department of Computer Science and Technology Huaqiao University Xiamen361021 China Xiamen Key Lab. of Computer Vision and Pattern Recognition Fujian Key Lab. of Big Data Intelligence and Security China Department of Computer Science Hong Kong Baptist University Kowloon Hong Kong School of Computer Science and Engineering University of Electronic Science and Technology of China China Provincial Key Laboratory for Computer Information Processing Technology Soochow University China

ISBN: (纸本)9781665438643

Existing cross-modal hashing still faces three challenges: (1) Most batch-based methods are unsuitable for processing large-scale and streaming data. (2) Current online methods often suffer from insufficient semantic association, while lacking flexibility to learn the hash functions for varying streaming data. (3) Existing supervised methods always require much computation time or accumulate large quantization loss to learn hash codes. To address above challenges, we present an efficient Online Label Consistent Hashing (OLCH) for cross-modal retrieval, which aims to incrementally learn hash codes for the current arriving data, while updating the hash functions at a streaming manner. To be specific, an online semantic representation learning framework is designed to adaptively preserve the semantic similarity across different modalities, and a mini-batch online gradient descent approach associated with forward-backward splitting is developed to optimize the hash functions. Accordingly, the hash codes are adaptively learned online with the high discriminative capability, while avoiding high computation complexity to process the streaming data. Experimental results show its outstanding performance in comparison with the-state-of-arts. © 2021 IEEE computer Society. All rights reserved.

关键词： Hash functions

Spectral, Probabilistic, and Deep Metric Learning: Tutorial and Survey

学校读者我要写书评

暂无评论

arXiv 2022年

作者： Ghojogh, Benyamin Ghodsi, Ali Karray, Fakhri Crowley, Mark Department of Electrical and Computer Engineering Machine Learning Laboratory University of Waterloo WaterlooON Canada Department of Statistics and Actuarial Science David R. Cheriton School of Computer Science Data Analytics Laboratory University of Waterloo WaterlooON Canada Department of Electrical and Computer Engineering Centre for Pattern Analysis and Machine Intelligence University of Waterloo WaterlooON Canada

This is a tutorial and survey paper on metric learning. Algorithms are divided into spectral, probabilistic, and deep metric learning. We first start with the definition of distance metric, Mahalanobis distance, and generalized Mahalanobis distance. In spectral methods, we start with methods using scatters of data, including the first spectral metric learning, relevant methods to Fisher discriminant analysis, Relevant Component Analysis (RCA), Discriminant Component Analysis (DCA), and the Fisher-HSIC method. Then, large-margin metric learning, imbalanced metric learning, locally linear metric adaptation, and adversarial metric learning are covered. We also explain several kernel spectral methods for metric learning in the feature space. We also introduce geometric metric learning methods on the Riemannian manifolds. In probabilistic methods, we start with collapsing classes in both input and feature spaces and then explain the neighborhood component analysis methods, Bayesian metric learning, information theoretic methods, and empirical risk minimization in metric learning. In deep learning methods, we first introduce reconstruction autoencoders and supervised loss functions for metric learning. Then, Siamese networks and its various loss functions, triplet mining, and triplet sampling are explained. Deep discriminant analysis methods, based on Fisher discriminant analysis, are also reviewed. Finally, we introduce multi-modal deep metric learning, geometric metric learning by neural networks, and few-shot metric learning. Copyright © 2022, The Authors. All rights reserved.

关键词： Surveys

Unraveling Attacks in machine Learning-based IoT Ecosystems: A Survey and the Open Libraries Behind Them

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Liu, Chao Chen, Boxi Shao, Wei Zhang, Chris Wong, Kelvin K.L. Zhang, Yi The Department of Electrical and Computer Engineering University of Maryland Baltimore CountyMD United States The Department of Research Deep Red Future Technology Co. Ltd. Shenzhen China The data61 CSIRO Australia The Department of Mechanical Engineering University of Saskatchewan Saskatchewan Canada Machine Intelligence Laboratory College of Computer Science Sichuan University Sichuan China

The advent of the Internet of Things (IoT) has brought forth an era of unprecedented connectivity, with an estimated 80 billion smart devices expected to be in operation by the end of 2025. These devices facilitate a multitude of smart applications, enhancing the quality of life and efficiency across various domains. machine Learning (ML) serves as a crucial technology, not only for analyzing IoT-generated data but also for diverse applications within the IoT ecosystem. For instance, ML finds utility in IoT device recognition, anomaly detection, and even in uncovering malicious activities. This paper embarks on a comprehensive exploration of the security threats arising from ML’s integration into various facets of IoT, spanning various attack types including membership inference, adversarial evasion, reconstruction, property inference, model extraction, and poisoning attacks. Unlike previous studies, our work offers a holistic perspective, categorizing threats based on criteria such as adversary models, attack targets, and key security attributes (confidentiality, availability, and integrity). We delve into the underlying techniques of ML attacks in IoT environment, providing a critical evaluation of their mechanisms and impacts. Furthermore, our research thoroughly assesses 65 libraries, both author-contributed and third-party, evaluating their role in safeguarding model and data privacy. We emphasize the availability and usability of these libraries, aiming to arm the community with the necessary tools to bolster their defenses against the evolving threat landscape. Through our comprehensive review and analysis, this paper seeks to contribute to the ongoing discourse on ML-based IoT security, offering valuable insights and practical solutions to secure ML models and data in the rapidly expanding field of artificial intelligence in IoT. © 2024, CC0.

关键词： Internet of things

Deep Learning in Palmprint Recognition-A Comprehensive Survey

学校读者我要写书评

暂无评论

arXiv 2025年

作者： Gao, Chengrui Yang, Ziyuan Jia, Wei Leng, Lu Zhang, Bob Teoh, Andrew Beng Jin College of Computer Science Sichuan University Chengdu610065 China Singapore School of Computer and Information Hefei University of Technology Hefei China Jiangxi Provincial Key Laboratory of Image Processing and Pattern Recognition Nanchang Hangkong University Nanchang China Pattern Analysis and Machine Intelligence Group Department of Computer and Information Science University of Macau Taipa China School of Electrical and Electronic Engineering College of Engineering Yonsei University Seoul Korea Republic of

Palmprint recognition has emerged as a prominent biometric technology, widely applied in diverse scenarios. Traditional handcrafted methods for palmprint recognition often fall short in representation capability, as they heavily depend on researchers' prior knowledge. Deep learning (DL) has been introduced to address this limitation, leveraging its remarkable successes across various domains. While existing surveys focus narrowly on specific tasks within palmprint recognition-often grounded in traditional methodologies-there remains a significant gap in comprehensive research exploring DL-based approaches across all facets of palmprint recognition. This paper bridges that gap by thoroughly reviewing recent advancements in DL-powered palmprint recognition. The paper systematically examines progress across key tasks, including region-of-interest segmentation, feature extraction, and security/privacy-oriented challenges. Beyond highlighting these advancements, the paper identifies current challenges and uncovers promising opportunities for future research. By consolidating state-of-the-art progress, this review serves as a valuable resource for researchers, enabling them to stay abreast of cutting-edge technologies and drive innovation in palmprint recognition. Copyright © 2025, The Authors. All rights reserved.

关键词： Biometrics