The integration of electric vehicles (EVs) into the smart grid has introduced new challenges and opportunities for optimizing power and energy management. This paper presents a simple method using a decision-tree to e...
详细信息
This research-to-practice paper presents a novel pedagogical tool for hardware cybersecurity education and workforce development. The growing importance of hardware security has made it essential for individuals and o...
详细信息
This paper investigates the application of Explainable AI (XAI) techniques in evaluating the features of indoor positioning systems, with a focus on improving model transparency and interpretability. Indoor positionin...
详细信息
System Strength (SS) is becoming increasingly important while more and more inverter-based resources (IBRs) get connected to weak parts of the grid. There is still, however, a level of ambiguity with respect to formal...
详细信息
The characterization of citrus fruits is crucial to optimize agricultural production and increase economic benefits for both producers and consumers. However, traditional characterization methods, such as chemical tec...
详细信息
Production machinery failures often cause significant financial losses, and traditional maintenance methods may struggle to mitigate these issues effectively. Predictive maintenance offers a modern solution by forecas...
详细信息
Methods in the field of quickest change detection rapidly detect in real-time a change in the data-generating distribution of an online data stream. Existing methods have been able to detect this change point when the...
详细信息
The need for aerial platforms capable of sustained operation is critical in fields such as surveillance, agricultural monitoring, disaster response, and temporary telecommunication systems. Traditional unmanned aerial...
详细信息
The load identification technologies in the domestic field, also called Non-Intrusive Load Monitoring (NILM), aim to break down the total electricity consumption at household level to the appliance level. This detaile...
详细信息
An increasingly popular machine learning paradigm is to pretrain a neural network (NN) on many tasks offline, then adapt it to downstream tasks, often by re-training only the last linear layer of the network. This app...
详细信息
An increasingly popular machine learning paradigm is to pretrain a neural network (NN) on many tasks offline, then adapt it to downstream tasks, often by re-training only the last linear layer of the network. This approach yields strong downstream performance in a variety of contexts, demonstrating that multitask pretraining leads to effective feature learning. Although several recent theoretical studies have shown that shallow NNs learn meaningful features when either (i) they are trained on a single task or (ii) they are linear, very little is known about the closer-to-practice case of nonlinear NNs trained on multiple tasks. In this work, we present the first results proving that feature learning occurs during training with a nonlinear model on multiple tasks. Our key insight is that multi-task pretraining induces a pseudo-contrastive loss that favors representations that align points that typically have the same label across tasks. Using this observation, we show that when the tasks are binary classification tasks with labels depending on the projection of the data onto an r-dimensional subspace within the d rdimensional input space, a simple gradient-based multitask learning algorithm on a two-layer ReLU NN recovers this projection, allowing for generalization to downstream tasks with sample and neuron complexity independent of d. In contrast, we show that with high probability over the draw of a single task, training on this single task cannot guarantee to learn all r ground-truth features. Copyright 2024 by the author(s)
暂无评论