Artificial neural networks are prone to suffer from catastrophic forgetting. Networks trained on something new tend to rapidly forget what was learned previously, a common phenomenon within connectionist models. In th...
详细信息
Artificial neural networks are prone to suffer from catastrophic forgetting. Networks trained on something new tend to rapidly forget what was learned previously, a common phenomenon within connectionist models. In this work, we propose an effective and efficient continual learning framework using random theory, together with Bayes' rule, to equip a single model with the ability to learn streaming data. The core idea of our framework is to preserve the performance of old tasks by guiding output weights to stay in a region of low error while encountering new tasks. In contrast to the existing continual learning approaches, our main contributions concern (1) closed-formed solutions with detailed theoretical analysis;(2) training continual learners by one-pass observation of samples;(3) remarkable advantages in terms of easy implementation, efficient parameters, fast convergence, and strong task-order robustness. Comprehensive experiments under popular image classification benchmarks, FashionMNIST, CIFAR-100, and imageNet, demonstrate that our methods predominately outperform the extensive state-of-the-art methods on training speed while maintaining superior accuracy and the number of parameters, in the class incremental learning scenario. Code is available at https://***/toil2sweet/CRNet.
The central idea of contrastive learning is to discriminate between different instances and force different views from the same instance to share the same representation. To avoid trivial solutions, augmentation plays...
详细信息
The central idea of contrastive learning is to discriminate between different instances and force different views from the same instance to share the same representation. To avoid trivial solutions, augmentation plays an important role in generating different views, among which random cropping is shown to be effective for the model to learn a generalized and robust representation. Commonly used random crop operation keeps the distribution of the difference between two views unchanged along the training process. In this work, we show that adaptively controlling the disparity between two augmented views along the training process enhances the quality of the learned representations. Specifically, we present a parametric cubic cropping operation, ParamCrop, for video contrastive learning, which automatically crops a 3D cubic by differentiable 3D affine transformations. ParamCrop is trained simultaneously with the video backbone using an adversarial objective, so that it learns to increase the contrastive loss and thus gradually reduces the shared contents between two cropped views. Experiments show that this adaptive and gradual increase in the disparity yielded by ParamCrop is beneficial to learning a strong and generalized representation for downstream tasks, which is shown to be effective on multiple contrastive learning frameworks and video backbones.
For output-coupled reaction-diffusion neural networks (RDNNs), the finite-time output synchronization (FTOS) issue is researched under directed topology in this paper. By virtue of the Lyapunov approach, inequality sk...
详细信息
For output-coupled reaction-diffusion neural networks (RDNNs), the finite-time output synchronization (FTOS) issue is researched under directed topology in this paper. By virtue of the Lyapunov approach, inequality skills and important lemmas, the feedback control protocol is designed to obtain the FTOS criterion for coupled RDNNs with output coupling. Furthermore, to ensure that the output-coupled RDNNs realize output synchronization in finite time, a novel adaptive coupling weights control strategy and output feedback controller are proposed. Finally, the relevant simulation results are presented to demonstrate the correctness of our obtained theorems respectively.
The resource-constrained project scheduling problem (RCPSP) is one of the project scheduling problems which are widely used in construction and many industrial disciplines. The challenge of the problem is to design so...
详细信息
The resource-constrained project scheduling problem (RCPSP) is one of the project scheduling problems which are widely used in construction and many industrial disciplines. The challenge of the problem is to design some appropriate search mechanism for finding solutions in feasible space. An improved genetic algorithm based on time window decomposition is proposed in this paper. Three derivation methods are applied to increase population diversity. The sampling count allocation strategy and the use of destructive lower bounds improve the search efficiency. The computational experiments on PSPLIB show that the proposed approach is more effective than that only using the decomposition mechanism and is competitive in solving two real-life cases. This research illustrates that continuously changing the search subspaces has potential advantages, which may be useful for studying RCPSP using other evolutionary algorithms in future. Some other better results may be obtained by using machine learning methods to flexibly determine the sampling times for each individual.
This paper presents a novel incremental consensus-based algorithm for solving a class of distributed optimization problems in multi-agent systems, considering input disturbances, equality constraints, and box constrai...
详细信息
This paper presents a novel incremental consensus-based algorithm for solving a class of distributed optimization problems in multi-agent systems, considering input disturbances, equality constraints, and box constraints. Traditional methods rely on average consensus to maintain the satisfaction of equality constraints throughout the entire evolution process. However, in practical applications, input disturbances can disrupt these equality constraints, rendering traditional methods ineffective. To address this challenge, the proposed algorithm combines integration sliding mode control technology with the observer methodology, creating a unified framework capable of handling input disturbances and preventing the system state from deviating beyond the solution space defined by the equality and box constraints. Moreover, the proposed algorithm offers the advantage of ensuring that all agents reach the optimal solution within a predefined time frame. This settling time can be directly adjusted by modifying one or more parameters. Finally, several numerical examples are validated to demonstrate the effectiveness and performance of the proposed algorithm. (c) 2025 Elsevier Ltd. All rights are reserved, including those for text and data mining, AI training, and similar technologies.
image matting, aiming to accurately extract foreground objects by estimating their opacity against the background, has made remarkable progress through deep-learning approaches. Nevertheless, the majority of these met...
详细信息
ISBN:
(数字)9798350349399
ISBN:
(纸本)9798350349405
image matting, aiming to accurately extract foreground objects by estimating their opacity against the background, has made remarkable progress through deep-learning approaches. Nevertheless, the majority of these methods require a user-defined auxiliary input, such as a trimap, which limits their applications in real-world scenarios. There are many auxiliary input-free methods that have been proposed by now, and some of them adopt a multi-task learning framework that includes a shared encoder and two separate decoders. However, these methods lack interactions between the two decoders, or interactions are implemented through simple summation or concatenation. Unfortunately, the integration of different features may cause negative transfer and limit the model performance due to the invisible information transmission process. To address the issue, we introduce the Pattern-Affinitive Propagation Module (PAP) to explicitly model cross-task propagation and task-specific propagation. Furthermore, image matting not only requires high-resolution detail features, but also semantic features. However, current CNN-based methods have limited receptive fields, making it challenging to capture global semantic features. Therefore, we design a module that integrates Dilated Convolution and Spectral Transformer (DSM), which can effectively capture global features and enhance global-local feature fusion. Extensive experiments on AM-2k and P3M-10k datasets demonstrate the superiority of our method.
Over the years, many state-of-the-art technologies have reinforced safety in the high-speed railway driving process. However, the accident rate has not dropped significantly with the advanced technology onboard. The l...
详细信息
Over the years, many state-of-the-art technologies have reinforced safety in the high-speed railway driving process. However, the accident rate has not dropped significantly with the advanced technology onboard. The leading cause of this phenomenon is adverse human performance. The crucial aspects contributing to human performance are performance shaping factors (PSFs). The existing expert judgment techniques evaluate human error probabilities depending on the descriptions of PSFs' influences on human performance. However, they cannot perform dynamic human error assessment with the real-time fluctuations of PSF interventions. To overcome this deficiency, we propose a deep learning -based method considering the instantaneous effect and time -dependent influence of PSF interventions on human performance and matching different neural network characteristics to human cognitive processes. The proposed SNN-STLSTM method combines the spiking neural network (SNN) auto -encoder and the sparse -temporal long short-term memory (STLSTM) network. The unsupervised SNN, a brain -like computational model, simulates the human brain's response when PSF interventions occur instantaneously. Meanwhile, an auto -encoder architecture integrating the unsupervised SNN makes the representations learned by SNN more meaningful. In addition, for the irregularity of PSF interventions, the standard LSTM is extended to the architecture STLSTM to predict the long-term effects of PSFs on human performance. Finally, a case study of the Wuhan-Guangzhou (WH-GZ) high-speed railway (HSR) shows how Monte Carlo simulation based on Fault Tree Analysis is used to assess and collect human error data and then how PSF and human error data are used to train the whole framework. The comparative experiment and ablation study indicate that the proposed SNN-STLSTM method outperforms other artificial intelligence approaches. This study provides insightful findings that help to understand how and what aspects affect h
This paper proposes a two-layer distributed network predictive control strategy for AC microgrids (MGs) clusters with communication delays. The strategy involves establishing a two-layer communication network to regul...
详细信息
This paper proposes a two-layer distributed network predictive control strategy for AC microgrids (MGs) clusters with communication delays. The strategy involves establishing a two-layer communication network to regulate the voltage/frequency of all distributed generators (DGs) within the MG cluster to predefined reference values while ensuring consistency in incremental costs across individual MGs. Furthermore, a multi-step predictive controller is designed, where delay information in the controller is replaced by the latest predictions, enabling proactive compensation for delays. Stability analysis of the closed-loop AC MG clusters is conducted and the response matching condition is derived between the second and tertiary levels. Finally, real-time simulations on an OPAL-RT platform are performed for AC MG clusters, validating the robustness of the proposed control method against communication delays.
This paper investigates the H-infinity performance of two-time-scale systems with nonlinear uncertainty and external disturbance via the event-triggered integral sliding mode control (ETISMC) strategy. To facilitate t...
详细信息
This paper investigates the H-infinity performance of two-time-scale systems with nonlinear uncertainty and external disturbance via the event-triggered integral sliding mode control (ETISMC) strategy. To facilitate the analysis of the sliding mode surface's reachability and the design of the event-triggered (ET) integral sliding mode controller, the epsilon-dependent integral sliding function with the event-triggered mechanism is constructed. Based on H-infinity control theory, the integral sliding mode controller gain and the ET parameter matrix are co-designed through the solution of a set of linear matrix inequalities. The proposed event-triggered mechanism can avoid Zeno behavior. Furthermore, an adaptive ETISMC strategy is proposed to address the unknown upper bounds of uncertainty and external disturbance. Finally, the effectiveness, applicability, and advantages of the control strategies are verified by simulations and comparative analysis.
The widespread adoption of advanced metering infrastructure has provided abundant data, enabling the integration of deep learning techniques into smart grids. However, it has also led to more sophisticated and conceal...
详细信息
The widespread adoption of advanced metering infrastructure has provided abundant data, enabling the integration of deep learning techniques into smart grids. However, it has also led to more sophisticated and concealed methods of electricity theft. Due to the challenges posed by data imbalance and missing values caused by device malfunctions and communication issues, existing deep learning models often perform poorly. To address these issues, this paper proposes a multi-step training framework named DING, which incorporates diffusion generation, self-supervised pre-training, normalized condition imputation, and generation-balanced fine-tuning. First, sufficient balanced smart meter data is generated using a diffusion model. Second, a pre-trained encoder is trained on the generated data, extracting unbiased low-dimensional features that can be used for downstream classification tasks and as conditions to guide the training of the imputation model. Next, an imputation model is trained based on a diffusion state-space equation. Finally, fine-tuning is performed on the balanced data. Experiments on a real dataset from the State Grid Corporation of China demonstrate that the proposed method outperforms previous models for both electricity theft detection and imputation tasks.
暂无评论