Decentralized Federated Learning (DFL) enables collaborative model training across multiple devices without relying on a central server, preserving data privacy and achieving full decentralization. However, optimizing...
详细信息
Decentralized Federated Learning (DFL) enables collaborative model training across multiple devices without relying on a central server, preserving data privacy and achieving full decentralization. However, optimizing hyperparameters (HPs) in DFL presents significant challenges due to system and statistical heterogeneity and the lack of centralized coordination. Existing Hyperparameter Optimization (HPO) methods for Federated Learning (FL) typically rely on a central server, making them unsuitable for fully decentralized environments. These methods face scalability issues, high communication and computation overhead, and limited adaptability to node-specific requirements. The need for serverless, fully decentralized HPO is particularly critical in scenarios demanding minimal communication and efficient resource usage. To address these challenges, we propose the single-pass Decentralized Federated Hyperparameter Optimization framework (DFed-HPO), which integrates three advanced HPO strategies for computation and communication-efficient HP optimization. DFed-HPO includes three hyperparameter aggregation mechanisms: MetaRegress Aggregator (MA), Consensus Aggregator (CA), and Fusion Aggregator (FA). MA uses meta-learning principles to predict the performance of new configurations with minimal communication, while CA and FA enhance optimization and aggregation based on model similarity and consensus-building among nodes. By conducting HPO in a single pass, DFed-HPO reduces communication overhead while achieving robust hyperparameter consensus. We validated DFed-HPO on MNIST and the Electricity dataset, demonstrating its potential in energy management applications, including green hydrogen production using renewable energy systems. Results show that DFed-HPO improves model performance, accelerates convergence, and adapts to non-IID data, offering a scalable and efficient solution for decentralized AI networks in resource-constrained settings.
When a new problem is given to be solved with high performance, deep learning needs large effort to tune the model hyperparameters including model architectures and training hyperparameters. Many previous works have t...
详细信息
ISBN:
(纸本)9781538662496
When a new problem is given to be solved with high performance, deep learning needs large effort to tune the model hyperparameters including model architectures and training hyperparameters. Many previous works have tried to tune the model hyperparameters automatically, but the algorithms only target on searching either model architectures or training hyperparameters. However, simultaneous optimization of the model architectures and the training hyperparameters works slow and falls into bad local minimums because of the search space enlarged by the high correlation between the two model hyperparameters. In this paper, we propose a novel algorithm to efficiently find the best set of model architectures and training hyperparameters. To efficiently handle the large search space, the proposed algorithm selectively utilizes the given training samples, while limiting the search space by a novel ensemble sampling method. Also, the evaluation time is further reduced by a novel termination mechanism. The accelerated computation of the proposed algorithm is validated by using complex image datasets, which shows the state-of-the-art performance with the 70:9% reduction of computational time.
With the popularity of wearable devices, human behavior recognition technology is becoming increasingly important in social surveillance, health monitoring, smart home, and traffic management. However, traditional hum...
详细信息
With the popularity of wearable devices, human behavior recognition technology is becoming increasingly important in social surveillance, health monitoring, smart home, and traffic management. However, traditional human behavior recognition methods rely too much on the subjective experience of managers in hyperparameter selection, resulting in an inefficient parameter optimization process. To address this problem, this paper proposes a long short-term memory (LSTM) neural network model based on a subtraction-average-based optimizer (SABO) for human behavior recognition in wearable devices. Compared to the traditional method, the SABO-LSTM model significantly improves the recognition accuracy by automatically finding the optimal hyperparameters, which proves its innovation and superiority in practical applications. To demonstrate the effectiveness of the method, four evaluation metrics, including F1 score, precision, recall, and accuracy, are used to validate it on the UCI-HAR dataset and the WISDM dataset, and control groups are introduced for comparison. The experimental results show that SABO-LSTM can accurately perform the human behavior recognition task with an accuracy of 98.84% and 96.37% on the UCI-HAR dataset and the WISDM dataset, respectively. In addition, the experimental model outperforms the control model on all four evaluation metrics and outperforms existing recognition methods in terms of accuracy.
暂无评论