Multitask learning (MTL) is a challenging puzzle, particularly in the realm of computer vision (CV). Setting up vanilla deep MTL requires either hard or soft parameter sharing schemes that employ greedy search to find...
详细信息
Multitask learning (MTL) is a challenging puzzle, particularly in the realm of computer vision (CV). Setting up vanilla deep MTL requires either hard or soft parameter sharing schemes that employ greedy search to find the optimal network designs. Despite its widespread application, the performance of MTL models is vulnerable to under-constrained parameters. In this article, we draw on the recent success of vision transformer (ViT) to propose a multitask representation learning method called multitask ViT (MTViT), which proposes a multiple branch transformer to sequentially process the image patches (i.e., tokens in transformer) that are associated with various tasks. Through the proposed cross-task attention (CA) module, a task token from each task branch is regarded as a query for exchanging information with other task branches. In contrast to prior models, our proposed method extracts intrinsic features using the built-in self-attention mechanism of the ViT and requires just linear time on memory and computation complexity, rather than quadratic time. Comprehensive experiments are carried out on two benchmark datasets, including NYU-Depth V2 (NYUDv2) and CityScapes, after which it is found that our proposed MTViT outperforms or is on par with existing convolutional neuralnetwork (CNN)-based MTL methods. In addition, we apply our method to a synthetic dataset in which task relatedness is controlled. Surprisingly, experimental results reveal that the MTViT exhibits excellent performance when tasks are less related.
We introduce SwitchPath, a novel stochastic activation function that enhances neuralnetwork exploration, performance, and generalization, by probabilistically toggling between the activation of a neuron and its negat...
详细信息
ISBN:
(纸本)9783031789762;9783031789779
We introduce SwitchPath, a novel stochastic activation function that enhances neuralnetwork exploration, performance, and generalization, by probabilistically toggling between the activation of a neuron and its negation. SwitchPath draws inspiration from the analogies between neuralnetworks and decision trees, and from the exploratory and regularizing properties of DropOut as well. Unlike Dropout, which intermittently reduces network capacity by deactivating neurons, SwitchPath maintains continuous activation, allowing networks to dynamically explore alternative information pathways while fully utilizing their capacity. Building on the concept of epsilon-greedy algorithms to balance exploration and exploitation, SwitchPath enhances generalization capabilities over traditional activation functions. The exploration of alternative paths happens during training without sacrificing computational efficiency. This paper presents the theoretical motivations, practical implementations, and empirical results, showcasing all the described advantages of SwitchPath over established stochastic activation mechanisms.
Dropout is an effective strategy for the regularization of deepneuralnetworks. Applying tabu to the units that have been dropped in the recent epoch and retaining them for training ensures diversification in dropout...
详细信息
ISBN:
(纸本)9783031301049;9783031301056
Dropout is an effective strategy for the regularization of deepneuralnetworks. Applying tabu to the units that have been dropped in the recent epoch and retaining them for training ensures diversification in dropout. In this paper, we improve the Tabu Dropout mechanism for training deepneuralnetworks in two ways. Firstly, we propose to use tabu tenure, or the number of epochs a particular unit will not be dropped. Different tabu tenures provide diversification to boost the training of deepneuralnetworks based on the search landscape. Secondly, we propose an adaptive tabu algorithm that automatically selects the tabu tenure based on the training performances through epochs. On several standard benchmark datasets, the experimental results show that the adaptive tabu dropout and tabu tenure dropout diversify and perform significantly better compared to the standard dropout and basic tabu dropout mechanisms.
Fuel cell degradation is one of the main challenges of hydrogen fuel cell vehicles, which can be solved by robust prediction techniques like machine learning. In this research, a specific Proton-exchange membrane fuel...
详细信息
Fuel cell degradation is one of the main challenges of hydrogen fuel cell vehicles, which can be solved by robust prediction techniques like machine learning. In this research, a specific Proton-exchange membrane fuel cell stack is considered, and the experimental data are imported to predict the future behavior of the stack. Besides, four different prediction neuralnetworkalgorithms are considered, and deepneuralnetwork is selected. Furthermore, Simcenter Amesim software is used with the ability of dynamic simulation to calculate real-time fuel consumption, fuel cell degradation, and engine performance. Finally, to better understand how fuel cell degradation affects fuel consumption and life cycle emission, lifecycle assessment as a potential tool is carried out using GREET software. The results show that a degraded Proton-exchange membrane fuel cell stack can result in an increase in fuel consumption by 14.32 % in the New European driving cycle and 13.9 % in the FTP-75 driving cycle. The Life Cycle Assessment analysis results show that fuel cell degradation has a significant effect on fuel consumption and total emission. The results show that a fuel cell with a predicted degradation will emit 26.4 % more CO2 emissions than a Proton-exchange membrane fuel cell without degradation.
暂无评论