An increasingly popular machine learning paradigm is to pretrain a neural network (NN) on many tasks offline, then adapt it to downstream tasks, often by re-training only the last linear layer of the network. This app...
详细信息
An increasingly popular machine learning paradigm is to pretrain a neural network (NN) on many tasks offline, then adapt it to downstream tasks, often by re-training only the last linear layer of the network. This approach yields strong downstream performance in a variety of contexts, demonstrating that multitask pretraining leads to effective feature learning. Although several recent theoretical studies have shown that shallow NNs learn meaningful features when either (i) they are trained on a single task or (ii) they are linear, very little is known about the closer-to-practice case of nonlinear NNs trained on multiple tasks. In this work, we present the first results proving that feature learning occurs during training with a nonlinear model on multiple tasks. Our key insight is that multi-task pretraining induces a pseudo-contrastive loss that favors representations that align points that typically have the same label across tasks. Using this observation, we show that when the tasks are binary classification tasks with labels depending on the projection of the data onto an r-dimensional subspace within the d rdimensional input space, a simple gradient-based multitask learning algorithm on a two-layer ReLU NN recovers this projection, allowing for generalization to downstream tasks with sample and neuron complexity independent of d. In contrast, we show that with high probability over the draw of a single task, training on this single task cannot guarantee to learn all r ground-truth features. Copyright 2024 by the author(s)
Previous articles on unsupervised skeleton-based action recognition primarily focused on strategies for utilizing features to drive model optimization through methods like contrastive learning and reconstruction. Howe...
详细信息
Indoor Navigation System (INS) supports seamless movement of objects within confined spaces in smart environments. In this paper, a novel INS that relies on ESP32-based Received Signal Strength Indication (RSSI) measu...
详细信息
This paper presents a hybrid precision network that combines binary and multi-bit layers for efficient 3D hand pose estimation on resource-constrained devices. By transforming the state-of-the-art HandFoldingNet (HFN)...
详细信息
Increasing the permissible operating speed of permanent magnet electric machines (PMSM) presents a range of intrinsic benefits. This however also raises a myriad of challenges associated with high-speed operation, inc...
详细信息
In order to improve the performance of contemporary and prospective technologies grounded in Network Function Virtualization (NFV), it is imperative to concentrate on identifying the most effective strategies for the ...
详细信息
The purpose of this article is to propose Stability-based Energy-Efficient Link-State Hybrid Routing(S-ELHR),a low latency routing proto-col that aims to provide a stable mechanism for routing in unmanned aerial vehic...
详细信息
The purpose of this article is to propose Stability-based Energy-Efficient Link-State Hybrid Routing(S-ELHR),a low latency routing proto-col that aims to provide a stable mechanism for routing in unmanned aerial vehicles(UAV).The S-ELHR protocol selects a number of network nodes to create a Connected Dominating Set(CDS)using a parameter known as the Stability Metric(SM).The SM considers the node’s energy usage,connectivity time,and node’s *** the highest SM nodes are chosen to form *** node declares a Willingness to indicate that it is prepared to serve as a relay for its neighbors,by employing its own energy state.S-ELHR is a hybrid protocol that stores only partial topological information and routing tables on CDS *** of relying on the routing information at each intermediary node,it uses source routing,in which a route is generated on-demand,and data packets contain the addresses of the nodes the packet will transit.A route recovery technique is additionally utilized,which first locates a new route to the destination before forwarding packets along *** simulation for various network sizes and mobility speeds,the efficiency of S-ELHR is *** findings demonstrate that S-ELHR performs better than Optimized Link State Routing(OLSR)and Energy Enhanced OLSR(EE-OLSR)in terms of packet delivery ratio,end-to-end delay,and energy consumption.
Nowadays, web applications play central roles in informationsystems using the Internet. Then, client-side web programming using HTML, CSS, and JavaScript should be mastered first by novice students. Previously, we ha...
详细信息
Inverter-based renewable energy resources, as a critical response to the net-zero energy transition, have been displacing conventional synchronous generation. However, this displacement has increased the grid's vu...
详细信息
暂无评论