检索结果-内蒙古大学图书馆

FedDA: Faster Framework of Local Adaptive Gradient Methods via Restarted Dual Averaging

学校读者我要写书评

暂无评论

arXiv 2023年

作者： Li, Junyi Huang, Feihu Huang, Heng Department of Electrical and Computer Engineering University of Pittsburgh Pittsburgh United States College of Computer Science and Technology Nanjing University of Aeronautics and Astronautics Nanjing China MIIT Key Laboratory of Pattern Analysis and Machine Intelligence Nanjing China

Federated learning (FL) is an emerging learning paradigm to tackle massively distributed data. In Federated Learning, a set of clients jointly perform a machine learning task under the coordination of a server. The FedAvg algorithm is one of the most widely used methods to solve Federated Learning problems. In FedAvg, the learning rate is a constant rather than changing adaptively. The adaptive gradient methods show superior performance over the constant learning rate schedule;however, there is still no general framework to incorporate adaptive gradient methods into the federated setting. In this paper, we propose FedDA, a novel framework for local adaptive gradient methods. The framework adopts a restarted dual averaging technique and is flexible with various gradient estimation methods and adaptive learning rate formulations. In particular, we analyze FedDA-MVR, an instantiation of our framework, and show that it achieves gradient complexity Õ(∊−1.5) and communication complexity Õ(∊−1) for finding a stationary point ∊. This matches the best known rate for first-order FL algorithms and FedDA-MVR is the first adaptive FL algorithm that achieves this rate. We also perform extensive numerical experiments to verify the efficacy of our method. Copyright © 2023, The Authors. All rights reserved.

关键词： Learning algorithms

Exploiting Saliency in Attention Based Convolutional Neural Network for Classification of Vertical Root Fractures 25th

学校读者我要写书评

暂无评论

Exploiting Saliency in Attention Based Convolutional Neural ...

25th International Conference on pattern Recognition Workshops, ICPR 2020

作者： Xu, Zhenxing Wan, Peng Aihemaiti, Gulibire Zhang, Daoqiang MIIT Key Laboratory of Pattern Analysis and Machine Intelligence College of Computer Science and Technology Nanjing University of Aeronautics and Astronautics Nanjing China Affiliated Stomatology Hospital of Medical School Nanjing University Nanjing China

ISBN: (纸本)9783030687625

Cone-beam computed tomography (CBCT) is widely used in clinical diagnosis of vertical root fractures (VRFs) which presents as crack on the teeth. However, manually checking the VRFs from a larger number of CBCT images is time-consuming and error-prone. Although the Convolutional Neural Networks (CNN) have achieved unprecedented progress in natural image recognition, end-to-end CNN is unsuitable to identify VRFs due to crack appears to be multi-scales and their complex relationships with surroundings tissues. We proposed a novel Feature Pyramids Attention Convolutional Neural Network (FPA-CNN), which incorporates saliency mask and multi-scale feature to boost the classification performance. Saliency map is viewed as spatial probability map where a person might look first to make a discriminative conclusion. Therefore it plays a role of high-level hint to guide the network focusing on the discriminative region. Experimental results demonstrate that our proposed FPA-CNN overcomes the challenge arised from multi-scale crack and complex contextual relationships. © 2021, Springer Nature Switzerland AG.

关键词： Cracks

MLC-NC: Long-Tailed Multi-Label Image Classification Through the Lens of Neural Collapse 39

学校读者我要写书评

暂无评论

MLC-NC: Long-Tailed Multi-Label Image Classification Through...

39th Annual AAAI Conference on Artificial intelligence, AAAI 2025

作者： Tao, Zijian Li, Shao-Yuan Wan, Wenhai Zheng, Jinpeng Chen, Jia-Yao Li, Yuchen Huang, Sheng-Jun Chen, Songcan MIIT Key Laboratory of Pattern Analysis and Machine Intelligence China College of Computer Science and Technology Nanjing University of Aeronautics and Astronautics China State Key Laboratory for Novel Software Technology Nanjing University China School of Computer Science and Technology Huazhong University of Science and Technology China College of Computer and Software Hohai University China

ISBN: (纸本)157735897X

Long-tailed (LT) data distribution is common in multi-label image classification (MLC) and can significantly impact the performance of classification models. One reason is the challenge of learning unbiased instance representations (i.e. features) for imbalanced datasets. Additionally, the co-occurrence of head/tail classes within the same instance, along with complex label dependencies, introduces further challenges. In this work, we delve into this problem through the lens of neural collapse (NC). NC refers to a phenomenon where the last-layer features and classifier of a deep neural network model exhibit a simplex Equiangular Tight Frame (ETF) structure during its terminal training phase. This structure creates an optimal linearly separable state. However, this phenomenon typically occurs in balanced datasets but rarely applies to the typical imbalanced problem. To induce NC properties under Long-tailed multi-label classification (LT-MLC) conditions, we propose an approach named MLC-NC, which aims to learn high-quality data representations and improve the model’s generalization ability. Specifically, MLC-NC accounts for the fact that different labels correspond to different feature parts located in images. MLC-NC extracts class-wise features from each instance through a cross-attention mechanism. To guide the features toward the ETF structure, we introduce visual-semantic feature alignment with a fixed ETF structured label embedding, which helps to learn evenly distributed class centers. To reduce within-class feature variation, we introduce collapse calibration within a lower-dimensional feature space. To mitigate classification bias, we concatenate features and feed them into a binarized fixed ETF classifier. As an orthogonal approach to existing methods, MLC-NC can be seamlessly integrated into various frameworks. Extensive experiments on widely-used benchmarks demonstrate the effectiveness of our method. Copyright © 2025, Association for the Advancement of Ar

关键词： Deep neural networks

Multi-Label Knowledge Distillation

学校读者我要写书评

暂无评论

Multi-Label Knowledge Distillation

International Conference on Computer Vision (ICCV)

作者： Penghui Yang Ming-Kun Xie Chen-Chen Zong Lei Feng Gang Niu Masashi Sugiyama Sheng-Jun Huang College of Computer Science and Technology Nanjing University of Aeronautics and Astronautics MIIT Key Laboratory of Pattern Analysis and Machine Intelligence Nanjing China School of Computer Science and Engineering Nanyang Technological University Singapore RIKEN Center for Advanced Intelligence Project The University of Tokyo Tokyo Japan

Existing knowledge distillation methods typically work by imparting the knowledge of output logits or intermediate feature maps from the teacher network to the student network, which is very successful in multi-class single-label learning. However, these methods can hardly be extended to the multi-label learning scenario, where each instance is associated with multiple semantic labels, because the prediction probabilities do not sum to one and feature maps of the whole example may ignore minor classes in such a scenario. In this paper, we propose a novel multi-label knowledge distillation method. On one hand, it exploits the informative semantic knowledge from the logits by dividing the multi-label learning problem into a set of binary classification problems; on the other hand, it enhances the distinctiveness of the learned feature representations by leveraging the structural information of label-wise embeddings. Experimental results on multiple benchmark datasets validate that the proposed method can avoid knowledge counteraction among labels, thus achieving superior performance against diverse comparing methods. Our code is available at: https://***/penghui-yang/L2D.

关键词：

Reconstruction Enhanced Multi-View Contrastive Learning for Anomaly Detection on Attributed Networks

学校读者我要写书评

暂无评论

arXiv 2022年

作者： Zhang, Jiaqiang Wang, Senzhang Chen, Songcan College of Computer Science and Technology Nanjing University of Aeronautics and Astronautics China MIIT Key Laboratory of Pattern Analysis and Machine Intelligence China Central South University China

Detecting abnormal nodes from attributed networks is of great importance in many real applications, such as financial fraud detection and cyber security. This task is challenging due to both the complex interactions between the anomalous nodes with other counterparts and their inconsistency in terms of attributes. This paper proposes a self-supervised learning framework that jointly optimizes a multi-view contrastive learning-based module and an attribute reconstruction-based module to more accurately detect anomalies on attributed networks. Specifically, two contrastive learning views are firstly established, which allow the model to better encode rich local and global information related to the abnormality. Motivated by the attribute consistency principle between neighboring nodes, a masked autoencoder-based reconstruction module is also introduced to identify the nodes which have large reconstruction errors, then are regarded as anomalies. Finally, the two complementary modules are integrated for more accurately detecting the anomalous nodes. Extensive experiments conducted on five benchmark datasets show our model outperforms current state-of-the-art models. Copyright © 2022, The Authors. All rights reserved.

关键词： Anomaly detection

Bayesian compressive principal component analysis

学校读者我要写书评

暂无评论

Frontiers of Computer Science 2020年第4期14卷 29-38页

作者： Di MA Songcan CHEN College of Computer Science and Technology Nanjing University of Aeronautics and AstronauticsNanjing211106China College of Computer Science and Technology Nanjing University of Aeronautics and AstronauticsMIIT Key Laboratory of Pattern Analysis and Machine IntelligenceNanjing211106China

Principal component analysis(PCA)is a widely used method for multivariate data analysis that projects the original high-dimensional data onto a low-dimensional subspace with maximum ***,in practice,we would be more likely to obtain a few compressed sensing(CS)measurements than the complete high-dimensional data due to the high cost of data acquisition and *** this paper,we propose a novel Bayesian algorithm for learning the solutions of PCA for the original data just from these CS *** this end,we utilize a generative latent variable model incorporated with a structure prior to model both sparsity of the original data and effective dimensionality of the latent *** proposed algorithm enjoys two important advantages:1)The effective dimensionality of the latent space can be determined automatically with no need to be pre-specified;2)The sparsity modeling makes us unnecessary to employ multiple measurement matrices to maintain the original data space but a single one,thus being storage *** results on synthetic and real-world datasets show that the proposed algorithm can accurately learn the solutions of PCA for the original data,which can in turn be applied in reconstruction task with favorable results.

关键词： compressed sensing principal component analysis Bayesian learning sparsity modeling

Bi-Grid Reconstruction for Image Anomaly Detection

学校读者我要写书评

暂无评论

arXiv 2025年

作者： Huang, Huichuan Zhong, Zhiqing Wei, Guangyu Wan, Yonghao Sun, Wenlong Feng, Aimin Nanjing University of Aeronautics and Astronautics China MIIT Key Laboratory of Pattern Analysis and Machine Intelligence China

—In image anomaly detection, significant advancements have been made using un- and self-supervised methods with datasets containing only normal samples. However, these approaches often struggle with fine-grained anomalies. This paper introduces GRAD: Bi-Grid Reconstruction for Image Anomaly Detection, which employs two continuous grids to enhance anomaly detection from both normal and abnormal perspectives. In this work: 1) Grids as feature repositories that improve generalization and mitigate the Identical Shortcut (IS) issue;2) An abnormal feature grid that refines normal feature boundaries, boosting detection of fine-grained defects;3) The Feature Block Paste (FBP) module, which synthesizes various anomalies at the feature level for quick abnormal grid deployment. GRAD’s robust representation capabilities also allow it to handle multiple classes with a single model. Evaluations on datasets like MVTecAD, VisA, and GoodsAD show significant performance improvements in fine-grained anomaly detection. GRAD excels in overall accuracy and in discerning subtle differences, demonstrating its superiority over existing methods. © 2025, CC BY-NC-ND.

关键词： Error detection

Multi-Label Knowledge Distillation

学校读者我要写书评

暂无评论

arXiv 2023年

作者： Yang, Penghui Xie, Ming-Kun Zong, Chen-Chen Feng, Lei Niu, Gang Sugiyama, Masashi Huang, Sheng-Jun College of Computer Science and Technology Nanjing University of Aeronautics and Astronautics China MIIT Key Laboratory of Pattern Analysis and Machine Intelligence Nanjing China School of Computer Science and Engineering Nanyang Technological University Singapore RIKEN Center for Advanced Intelligence Project Japan The University of Tokyo Tokyo Japan

Existing knowledge distillation methods typically work by imparting the knowledge of output logits or intermediate feature maps from the teacher network to the student network, which is very successful in multi-class single-label learning. However, these methods can hardly be extended to the multi-label learning scenario, where each instance is associated with multiple semantic labels, because the prediction probabilities do not sum to one and feature maps of the whole example may ignore minor classes in such a scenario. In this paper, we propose a novel multi-label knowledge distillation method. On one hand, it exploits the informative semantic knowledge from the logits by dividing the multi-label learning problem into a set of binary classification problems;on the other hand, it enhances the distinctiveness of the learned feature representations by leveraging the structural information of label-wise embeddings. Experimental results on multiple benchmark datasets validate that the proposed method can avoid knowledge counteraction among labels, thus achieving superior performance against diverse comparing methods. Our code is available at: https://***/penghui-yang/L2D. Copyright © 2023, The Authors. All rights reserved.

关键词： Distillation

Cprnc: Channels Pruning Via Reverse Neuron Crowding for Model Compression

学校读者我要写书评

暂无评论

SSRN

SSRN 2023年

作者： Wu, Pingfan Huang, Hengyi Sun, Han Liang, Dong Liu, Ningzhong College of Computer Science and Technology Nanjing University of Aeronautics and Astronautics Nanjing211106 China MIIT Key Laboratory of Pattern Analysis and Machine Intelligence Nanjing211106 China Collaborative Innovation Center of Novel Software Technology and Industrialization Nanjing211106 China

Channel pruning is an efficient technique for model compression, removing redundant parts of a convolutional neural network with minor degradation in classification accuracy. Previous criteria of channel pruning ignore neurons' intrinsic relationship and the high correlation with input samples. Inspired by the visual crowding phenomenon in neuroscience, this paper presents a novel channel pruning method via reverse neuron crowding, dubbed CPRNC, to address this issue. First, CPRNC involves a neuron crowding degree measure (NCDM) module, which builds the relationship model among all artificial neurons by observing their crowding behaviors. Subsequently, each channel's importance is evaluated by the crowding degree of corresponding channels. Considering that the channel importance is affected by the characteristic of input samples, CPRNC designs a neuron crowding degree recalibrate (NCDR) module. NCDR emphasizes discriminative samples to recalibrate the channel priority list generated by NCDM, further enhancing the precision of the pruning criterion. Experimental results show that CPRNC achieves performance that competes with state-of-the-art pruning methods, including dynamic channel pruning and learning-based pruning. For example, we prune ResNet-50 with 56.7% FLOPs on the large-scale dataset ImageNet1K with only a 0.19% decrease in accuracy. At low pruning rates, CPRNC achieves lossless compression, e.g., the pruned ResNet-56 on CIFAR-10 increases accuracy by 0.13% over the baseline model at 56.3% FLOPs reduction. © 2023, The Authors. All rights reserved.

关键词： Neurons