检索结果-内蒙古大学图书馆

Faster Adaptive Decentralized Learning Algorithms

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Huang, Feihu Zhao, Jianyu College of Computer Science and Technology Nanjing University of Aeronautics and Astronautics Nanjing China MIIT Key Laboratory of Pattern Analysis and Machine Intelligence Nanjing China

Decentralized learning recently has received increasing attention in machine learning due to its advantages in implementation simplicity and system robustness, data privacy. Meanwhile, the adaptive gradient methods show superior performances in many machine learning tasks such as training neural networks. Although some works focus on studying decentralized optimization algorithms with adaptive learning rates, these adaptive decentralized algorithms still suffer from high sample complexity. To fill these gaps, we propose a class of faster adaptive decentralized algorithms (i.e., AdaMDOS and AdaMDOF) for distributed nonconvex stochastic and finite-sum optimization, respectively. Moreover, we provide a solid convergence analysis framework for our methods. In particular, we prove that our AdaMDOS obtains a near-optimal sample complexity of Õ(ϵ−3) for finding an ϵ-stationary solution of nonconvex stochastic optimization. Meanwhile, our AdaMDOF obtains a near-optimal sample complexity of O(√nϵ−2) for finding an ϵ-stationary solution of nonconvex finite-sum optimization, where n denotes the sample size. To the best of our knowledge, our AdaMDOF algorithm is the first adaptive decentralized algorithm for nonconvex finite-sum optimization. Some experimental results demonstrate efficiency of our algorithms. © 2024, CC BY-NC-ND.

关键词： Optimization algorithms

Beyond myopia: learning from positive and unlabeled data through holistic predictive trends 23

学校读者我要写书评

暂无评论

Beyond myopia: learning from positive and unlabeled data thr...

Proceedings of the 37th International Conference on Neural Information Processing Systems

作者： Xinrui Wang Wenhai Wan Chuanxing Geng Shaoyuan Li Songcan Chen College of Computer Science and Technology Nanjing University of Aeronautics and Astronautics MIIT Key Laboratory of Pattern Analysis and Machine Intelligence

Learning binary classifiers from positive and unlabeled data (PUL) is vital in many real-world applications, especially when verifying negative examples is difficult. Despite the impressive empirical performance of recent PUL methods, challenges like accumulated errors and increased estimation bias persist due to the absence of negative labels. In this paper, we unveil an intriguing yet long-overlooked observation in PUL: resampling the positive data in each training iteration to ensure a balanced distribution between positive and unlabeled examples results in strong early-stage performance. Furthermore, predictive trends for positive and negative classes display distinctly different patterns. Specifically, the scores (output probability) of unlabeled negative examples consistently decrease, while those of unlabeled positive examples show largely chaotic trends. Instead of focusing on classification within individual time frames, we innovatively adopt a holistic approach, interpreting the scores of each example as a temporal point process (TPP). This reformulates the core problem of PUL as recognizing trends in these scores. We then propose a novel TPP-inspired measure for trend detection and prove its asymptotic unbiasedness in predicting changes. Notably, our method accomplishes PUL without requiring additional parameter tuning or prior assumptions, offering an alternative perspective for tackling this problem. Extensive experiments verify the superiority of our method, particularly in a highly imbalanced real-world setting, where it achieves improvements of up to 11.3% in key metrics. The code is available at https://***/wxr99/HolisticPU.

关键词：

A generative deep learning framework for airfoil flow field prediction with sparse data

学校读者我要写书评

暂无评论

Chinese Journal of Aeronautics 2022年第1期35卷 470-484页

作者： Haizhou WU Xuejun LIU Wei AN Hongqiang LYU MIIT Key Laboratory of Pattern Analysis and Machine Intelligence College of Computer Science and TechnologyNanjing University of Aeronautics and AstronauticsNanjing 211106China Key Laboratory of Aerodynamic Noise Control Mianyang 621000China State Key Laboratory of Aerodynamics Mianyang 621000China Collaborative Innovation Center of Novel Software Technology and Industrialization Nanjing 210023China College of Aerospace Engineering Nanjing University of Aeronautics and AstronauticsNanjing 211106China

Deep learning has been probed for the airfoil performance prediction in recent *** with the expensive CFD simulations and wind tunnel experiments,deep learning models can be leveraged to somewhat mitigate such expenses with proper ***,effective training of the data-driven models in deep learning severely hinges on the data in diversity and *** this paper,we present a novel data augmented Generative Adversarial Network(GAN),daGAN,for rapid and accurate flow filed prediction,allowing the adaption to the task with sparse *** presented approach consists of two modules,pre-training module and fine-tuning *** pre-training module utilizes a conditional GAN(cGAN)to preliminarily estimate the distribution of the training *** the fine-tuning module,we propose a novel adversarial architecture with two generators one of which fulfils a promising data augmentation operation,so that the complement data is adequately incorporated to boost the generalization of the *** use numerical simulation data to verify the generalization of daGAN on airfoils and flow conditions with sparse training *** results show that daGAN is a promising tool for rapid and accurate evaluation of detailed flow field without the requirement for big training data.

关键词： CFD Flow field Generative adversarial networks(GANs) Sparse data Supercritical airfoil

Class-Aware Universum Inspired Re-Balance Learning for Long-Tailed Recognition

学校读者我要写书评

暂无评论

SSRN

SSRN 2024年

作者： Zhang, Enhao Geng, Chuanxing Chen, Songcan College of Computer Science and Technology Nanjing University of Aeronautics and Astronautics Jiangsu Nanjing211106 China MIIT Key Laboratory of Pattern Analysis and Machine Intelligence China

Data augmentation for minority classes is an effective strategy for long-tailed recognition, thus developing a large number of methods. Although these methods all ensure the balance in sample quantity, the quality of the augmented samples is not always satisfactory for recognition, being prone to such problems as over-fitting, lack of diversity, semantic drift, etc. For these issues, we propose the Class-aware Universum Inspired Re-balance Learning (CaUIRL) for long-tailed recognition, which endows the Universum with class-aware ability to re-balance individual minority classes from both sample quantity and quality. In particular, we theoretically prove that the classifiers learned by CaUIRL are consistent with those learned under the balanced condition from a Bayesian perspective. In addition, we further develop a higher-order mixup approach, which can automatically generate class-aware Universum (CaU) data without resorting to any external data. Unlike the traditional Universum, such generated Universum additionally takes the domain similarity, class separability, and sample diversity into account. Extensive experiments on benchmark datasets demonstrate surprising advantages of our method over state-of-the-art methods, especially in minority classes (e.g., the top-1 accuracy of the last two classes of the tail of Cifar10-LR is improved by 6%). © 2024, The Authors. All rights reserved.

关键词： Semantics

Beyond Myopia: Learning from Positive and Unlabeled Data through Holistic Predictive Trends

学校读者我要写书评

暂无评论

arXiv 2023年

作者： Wang, Xinrui Wan, Wenhai Geng, Chuanxin Li, Shaoyuan Chen, Songcan College of Computer Science and Technology Nanjing University of Aeronautics Astronautics MIIT Key Laboratory of Pattern Analysis and Machine Intelligence China

关键词： Iterative methods

Implicit Stochastic Gradient Descent for Training Physics-informed Neural Networks

学校读者我要写书评

暂无评论

arXiv 2023年

作者： Li, Ye Chen, Song-Can Huang, Sheng-Jun College of Computer Science and Technology/Artificial Intelligence Nanjing University of Aeronautics Astronautics MIIT Key Laboratory of Pattern Analysis and Machine Intelligence Nanjing China

Physics-informed neural networks (PINNs) have effectively been demonstrated in solving forward and inverse differential equation problems, but they are still trapped in training failures when the target functions to be approximated exhibit high-frequency or multi-scale features. In this paper, we propose to employ implicit stochastic gradient descent (ISGD) method to train PINNs for improving the stability of training process. We heuristically analyze how ISGD overcome stiffness in the gradient flow dynamics of PINNs, especially for problems with multi-scale solutions. We theoretically prove that for two-layer fully connected neural networks with large hidden nodes, randomly initialized ISGD converges to a globally optimal solution for the quadratic loss function. Empirical results demonstrate that ISGD works well in practice and compares favorably to other gradient-based optimization methods such as SGD and Adam, while can also effectively address the numerical stiffness in training dynamics via gradient descent. Copyright © 2023, The Authors. All rights reserved.

关键词： Stiffness

Decoding the Echoes of Vision from fMRI: Memory Disentangling for Past Semantic Information

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Xia, Runze Yin, Congchi Li, Piji 1 College of Computer Science and Technology Nanjing University of Aeronautics and Astronautics China 2 MIIT Key Laboratory of Pattern Analysis and Machine Intelligence Nanjing China

The human visual system is capable of processing continuous streams of visual information, but how the brain encodes and retrieves recent visual memories during continuous visual processing remains unexplored. This study investigates the capacity of working memory to retain past information under continuous visual stimuli. And then we propose a new task Memory Disentangling, which aims to extract and decode past information from fMRI signals. To address the issue of interference from past memory information, we design a disentangled contrastive learning method inspired by the phenomenon of proactive interference. This method separates the information between adjacent fMRI signals into current and past components and decodes them into image descriptions. Experimental results demonstrate that this method effectively disentangles the information within fMRI signals. This research could advance brain-computer interfaces and mitigate the problem of low temporal resolution in fMRI. 1 Copyright © 2024, The Authors. All rights reserved.

关键词： Contrastive Learning

Cross-Subject Data Splitting for Brain-to-Text Decoding

学校读者我要写书评

暂无评论

arXiv 2023年

作者： Yin, Congchi Yu, Qian Fang, Zhiwei He, Jie Peng, Changping Lin, Zhangang Shao, Jingping Li, Piji Nanjing University of Aeronautics and Astronautics China MIIT Key Laboratory of Pattern Analysis and Machine Intelligence Nanjing China *** Beijing China

Recent major milestones have successfully decoded non-invasive brain signals (e.g. functional Magnetic Resonance Imaging (fMRI) and electroencephalogram (EEG)) into natural language. Despite the progress in model design, how to split the datasets for training, validating, and testing still remains a matter of debate. Most of the prior researches applied subject-specific data splitting, where the decoding model is trained and evaluated per subject. Such splitting method poses challenges to the utilization efficiency of dataset as well as the generalization of models. In this study, we propose a cross-subject data splitting criterion for brain-to-text decoding on various types of cognitive dataset (fMRI, EEG), aiming to maximize dataset utilization and improve model generalization. We undertake a comprehensive analysis on existing cross-subject data splitting strategies and prove that all these methods suffer from data leakage, namely the leakage of test data to training set, which significantly leads to overfitting and overestimation of decoding models. The proposed cross-subject splitting method successfully addresses the data leakage problem and we re-evaluate some SOTA brain-to-text decoding models as baselines for further research. © 2023, CC BY.

关键词： Electroencephalography

An Improved AdaBoost Method in Imbalanced Data Learning

学校读者我要写书评

暂无评论

An Improved AdaBoost Method in Imbalanced Data Learning

Cyber-Physical Social intelligence (ICCSI), International Conference on

作者： Ting Li Xiaofeng Chen Weikai Li Department of Mathematics Chongqing Jiaotong University Chongqing China MIIT Key Laboratory of Pattern Analysis and Machine Intelligence Nanjing China

Learning from imbalanced datasets has recently received increasing research attention. Despite the remarkable results of AdaBoost in balanced situation, the imbalance problem remains to be solved. To address this, this paper proposes a novel method to improve the AdaBoost using a new error factor and a weighted voting parameter for a few classes and weak classifiers. Our method can weaken the dominant role of the majority class in iterative training by improving the classification ability of the minority class. The superiority of the proposed algorithm is finally proved theoretically and practically. Several real-world imbalance datasets in experiments as specific support, and our method outperforms previous algorithms, especially in the F-1 metric.

关键词：