检索结果-内蒙古大学图书馆

Compressed Gradient Tracking Algorithms for distributed Nonconvex Optimization

学校读者我要写书评

暂无评论

arXiv 2023年

作者： Xu, Lei Yi, Xinlei Wen, Guanghui Shi, Yang Johansson, Karl H. Yang, Tao The State Key Laboratory of Synthetical Automation for Process Industries Northeastern University Shenyang110819 China The Lab for Information & Decision Systems Massachusetts Institute of Technology CambridgeMA02139 United States The Department of Systems Science School of Mathematics Southeast University Nanjing210096 China The Department of Mechanical Engineering University of Victoria VictoriaBCV8W 2Y2 Canada The Division of Decision and Control Systems School of Electrical Engineering and Computer Science KTH Royal Institute of Technology Stockholm100 44 Sweden

In this paper, we study the distributed nonconvex optimization problem, aiming to minimize the average value of the local nonconvex cost functions using local information exchange. To reduce the communication overhead, we introduce three general classes of compressors, i.e., compressors with bounded relative compression error, compressors with globally bounded absolute compression error, and compressors with locally bounded absolute compression error. By integrating them, respectively, with the distributed gradient tracking algorithm, we then propose three corresponding compressed distributed nonconvex optimization algorithms. Motivated by the state-of-the-art BEER algorithm proposed in [1], which is an efficient compressed algorithm integrating gradient tracking with biased and contractive compressors, our first proposed algorithm extends this algorithm to accommodate both biased and non-contractive compressors. For each algorithm, we design a novel Lyapunov function to demonstrate its sublinear convergence to a stationary point if the local cost functions are smooth. Furthermore, when the global cost function satisfies the Polyak–Lojasiewicz (P–L) condition, we show that our proposed algorithms linearly converge to a global optimal point. It is worth noting that, for compressors with bounded relative compression error and globally bounded absolute compression error, our proposed algorithms’ parameters do not require prior knowledge of the P–L constant. The theoretical results are illustrated by numerical examples, demonstrating the effectiveness of the proposed algorithms in significantly reducing the communication burden while maintaining the convergence performance. Moreover, simulation results show that the proposed algorithms outperform the state-of-the-art compressed distributed nonconvex optimization algorithms. Copyright © 2023, The Authors. All rights reserved.

关键词： Cost functions

Global solution to sensor network localization: A non-convex potential game approach and its distributed implementation

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Xu, Gehui Chen, Guanpu Hong, Yiguang Fidan, Baris Parisini, Thomas Johansson, Karl H. Key Laboratory of Systems and Control Academy of Mathematics and Systems Science Beijing100190 China The Division of Decision and Control Systems School of Electrical Engineering and Computer Science KTH Royal Institute of Technology Sweden Digital Futures StockholmSE-10044 Sweden Department of Control Science and Engineering Tongji University Shanghai201804 China Shanghai Research Institute for Intelligent Autonomous Systems Shanghai201210 China The Department of Mechanical and Mechatronics Engineering University of Waterloo WaterlooONN2L 3G1 Canada The Department of Electrical and Electronic Engineering Imperial College London LondonSW7 2AZ United Kingdom The Department of Engineering and Architecture University of Trieste Trieste34127 Italy

Consider a sensor network consisting of both anchor and non-anchor nodes. We address the following sensor network localization (SNL) problem: given the physical locations of anchor nodes and relative measurements among all nodes, determine the locations of all non-anchor nodes. The solution to the SNL problem is challenging due to its inherent non-convexity. In this paper, the problem takes on the form of a multi-player nonconvex potential game in which canonical duality theory is used to define a complementary dual potential function. After showing the Nash equilibrium (NE) correspondent to the SNL solution, we provide a necessary and sufficient condition for a stationary point to coincide with the NE. An algorithm is proposed_ to reach the NE and shown to have convergence rate O(1/√k). With the aim of reducing the information exchange within a network, a distributed algorithm for NE seeking is implemented and its global convergence analysis is provided. Extensive simulations show the validity and effectiveness of the proposed approach to solve the SNL problem. Copyright © 2024, The Authors. All rights reserved.

关键词： Game theory

Deficient Excitation in Parameter Learning

学校读者我要写书评

暂无评论

arXiv 2025年

作者： Cao, Ganghui Wang, Shimin Guay, Martin Wang, Jinzhi Duan, Zhisheng Polycarpou, Marios M. State Key Laboratory for Turbulence and Complex Systems Department of Mechanics and Engineering Science College of Engineering Peking University Beijing100871 China Massachusetts Institute of Technology CambridgeMA02139 United States Queen’s University KingstonONK7L 3N6 Canada KIOS Research and Innovation Center of Excellence Department of Electrical and Computer Engineering University of Cyprus Nicosia1678 Cyprus

This paper investigates parameter learning problems under deficient excitation (DE). The DE condition is a rank-deficient, and therefore, a more general evolution of the well-known persistent excitation condition. Under the DE condition, a proposed online algorithm is able to calculate the identifiable and non-identifiable subspaces, and finally give an optimal parameter estimate in the sense of least squares. In particular, the learning error within the identifiable subspace exponentially converges to zero in the noise-free case, even without persistent excitation. The DE condition also provides a new perspective for solving distributed parameter learning problems, where the challenge is posed by local regressors that are often insufficiently excited. To improve knowledge of the unknown parameters, a cooperative learning protocol is proposed for a group of estimators that collect measured information under complementary DE conditions. This protocol allows each local estimator to operate locally in its identifiable subspace, and reach a consensus with neighbours in its non-identifiable subspace. As a result, the task of estimating unknown parameters can be achieved in a distributed way using cooperative local estimators. Application examples in system identification are given to demonstrate the effectiveness of the theoretical results developed in this paper. Copyright © 2025, The Authors. All rights reserved.

关键词： Contrastive Learning

HUNTER: AI based holistic resource management for sustainable cloud computing

学校读者我要写书评

暂无评论

arXiv 2021年

作者： Tuli, Shreshth Gill, Sukhpal Singh Xu, Minxian Garraghan, Peter Bahsoon, Rami Prof, Scharam Dustdar Sakellariou, Rizos Rana, Omer Buyya, Rajkumar Casale, Giuliano Jennings, Nicholas R. Department of Computing Imperial College London United Kingdom School of Electronic Engineering and Computer Science Queen Mary University of London United Kingdom Shenzhen Institutes of Advanced Technology Chinese Academy of Sciences China School of Computing and Communications Lancaster University United Kingdom School of Computer Science University of Birmingham United Kingdom Distributed Systems Group Vienna University of Technology Austria Department of Computer Science University of Manchester United Kingdom School of Computer Science and Informatics Cardiff University United Kingdom Laboratory School of Computing and Information Systems The University of Melbourne Australia

The worldwide adoption of cloud data centers (CDCs) has given rise to the ubiquitous demand for hosting application services on the cloud. Further, contemporary data-intensive industries have seen a sharp upsurge in the resource requirements of modern applications. This has led to the provisioning of an increased number of cloud servers, giving rise to higher energy consumption and, consequently, sustainability concerns. Traditional heuristics and reinforcement learning based algorithms for energy-efficient cloud resource management address the scalability and adaptability related challenges to a limited extent. Existing work often fails to capture dependencies across thermal characteristics of hosts, resource consumption of tasks and the corresponding scheduling decisions. This leads to poor scalability and an increase in the compute resource requirements, particularly in environments with non-stationary resource demands. To address these limitations, we propose an artificial intelligence (AI) based holistic resource management technique for sustainable cloud computing called HUNTER. The proposed model formulates the goal of optimizing energy efficiency in data centers as a multi-objective scheduling problem, considering three important models: energy, thermal and cooling. HUNTER utilizes a Gated Graph Convolution Network as a surrogate model for approximating the Quality of Service (QoS) for a system state and generating optimal scheduling decisions. Experiments on simulated and physical cloud environments using the CloudSim toolkit and the COSCO framework show that HUNTER outperforms state-of-the-art baselines in terms of energy consumption, SLA violation, scheduling time, cost and temperature by up to 12, 35, 43, 54 and 3 percent respectively. © 2021, CC BY.

关键词： Energy efficiency

distributed Task-Oriented Communication Networks with Multimodal Semantic Relay and Edge Intelligence

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Guo, Jie Chen, Hao Song, Bin Chi, Yuhao Yuen, Chau Yu, Fei Richard Li, Geoffrey Ye Niyato, Dusit The State Key Laboratory of Integrated Services Networks Xidian University Xi’an710071 China The Hangzhou Institute of Technology Xidian University Hangzhou311231 China The School of Electrical and Electronics Engineering The School of Computer Science and Engineering Respectively Nanyang Technological University 639798 Singapore The Department of Systems and Computer Engineering Carleton University OttawaK1S 5B6 Canada The Faculty of Engineering Imperial College London LondonSW7 2AZ United Kingdom

In this article, we present a novel framework, named distributed task-oriented communication networks (DTCN), based on recent advances in multimodal semantic transmission and edge intelligence. In DTCN, the multimodal knowledge of semantic relays and the adaptive adjustment capability of edge intelligence can be integrated to improve task performance. Specifically, we propose the key techniques in the framework, such as semantic alignment and complement, a semantic relay scheme for deep joint source-channel relay coding, and collaborative device-server optimization and inference. Furthermore, a multimodal classification task is used as an example to demonstrate the benefits of the proposed DTCN over existing methods. Numerical results validate that DTCN can significantly improve the accuracy of classification tasks, even in harsh communication scenarios (e.g., low signal-to-noise regime), thanks to multimodal semantic relay and edge intelligence. © 2024, CC BY-NC-SA.

关键词： Signal to noise ratio

Anomaly Detection Via Kpis for Software Performance Failures

学校读者我要写书评

暂无评论

SSRN

SSRN 2022年

作者： Wang, Shuguang Lu, Minyan Kong, Shiyi Ai, Jun Wang, Jie Wong, W. Eric The Key Laboratory on Reliability and Environment Engineering Technology School of Reliability and Systems Engineering Beihang University Beijing100191 China Department of Computer Science University of Texas at Dallas 2601 N Floyd Road RichardsonTX75083 United States

As cloud services have become more popular, high-reliability requirements for distributed storage systems have also increased significantly. However, the software systems often suffer performance failures during long-term running. Anomaly detection based on performance indicators is an effective approach for performance failures. To handle the complexity of the distributed databases, this paper proposes a novel anomaly detection methodology via key performance indicators monitored at different levels. Then, a variance inflation factor (VIF) feature selection method is proposed to reduce the multicollinearity problem in the multivariate time series of performance indicators. For a long-running software system, some unavoidable noise will be introduced when monitoring these performance indicators. Therefore, it is necessary to explore the suitable feature processing method to enhance features and eliminate noise interference from indicators. Discrete wavelet analysis has a superior performance in feature extraction in signal analysis. Thus, in this paper, discrete wavelet transform is modified to create an adaptive DWT for noise reduction and feature extraction by taking advantage of the multiresolution analysis. Then, unsupervised anomaly detection algorithms are used to detect the expected indicator outliers. The anomaly detection performance of the proposed method is evaluated in terms of precision, recall, and the F1-score. Finally, this approach is verified through the experiments conducted on Cassandra, and the F1-score is 0.974 which means this approach is effective. © 2022, The Authors. All rights reserved.

关键词： Benchmarking

NUMA-aware FFT-based convolution on ARMv8 many-core CPUs

学校读者我要写书评

暂无评论

arXiv 2021年

作者： Huang, Xiandong Wang, Qinglin Lu, Shuyu Hao, Ruochen Mei, Songzhu Liu, Jie Science and Technology on Parallel and Distributed Processing Laboratory National University of Defense Technology Changsha410073 China School of Computer Science National University of Defense Technology Changsha410073 China Department of Biomedical Informatics University of Pittsburgh Pittsburgh United States

Convolutional Neural Networks (CNNs), one of the most representative algorithms of deep learning, are widely used in various artificial intelligence applications. Convolution operations often take most of the computational overhead of CNNs. The FFT-based algorithm can improve the efficiency of convolution by reducing its algorithm complexity, there are a lot of works about the high-performance implementation of FFT-based convolution on many-core CPUs. However, there is no optimization for the non-uniform memory access (NUMA) characteristics in manycore CPUs. In this paper, we present a NUMA-aware FFT-based convolution implementation on ARMv8 many-core CPUs with NUMA architectures. The implementation can reduce a number of remote memory access through the data reordering of FFT transformations and the three-level parallelization of the complex matrix multiplication. The experiment results on a ARMv8 many-core CPU with NUMA architectures demonstrate that our NUMA-aware implementation has much better performance than the state-of-the-art work in most cases. Copyright © 2021, The Authors. All rights reserved.

关键词： Convolutional neural networks

A Survey of Recent Advances in Optimization Methods for Wireless Communications

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Liu, Ya-Feng Chang, Tsung-Hui Hong, Mingyi Wu, Zheyu So, Anthony Man-Cho Jorswieck, Eduard A. Yu, Wei The State Key Laboratory of Scientific and Engineering Computing Institute of Computational Mathematics and Scientific/Engineering Computing Academy of Mathematics and Systems Science Chinese Academy of Sciences Beijing100190 China The School of Science and Engineering The Chinese University of Hong Kong Shenzhen Shenzhen China Shenzhen Research Institute of Big Data China The Department of Electrical and Computer Engineering University of Minnesota MinneapolisMN55455 United States The Department of Systems Engineering and Engineering Management The Chinese University of Hong Kong Hong Kong The Institute for Communication Technology Technische Universitat Braunschweig Germany The Edward S. Rogers Sr. Department of Electrical and Computer Engineering University of Toronto TorontoONM5S 3G4 Canada

Mathematical optimization is now widely regarded as an indispensable modeling and solution tool for the design of wireless communications systems. While optimization has played a significant role in the revolutionary progress in wireless communication and networking technologies from 1G to 5G and onto the future 6G, the innovations in wireless technologies have also substantially transformed the nature of the underlying mathematical optimization problems upon which the system designs are based and have sparked significant innovations in the development of methodologies to understand, to analyze, and to solve those problems. In this paper, we provide a comprehensive survey of recent advances in mathematical optimization theory and algorithms for wireless communication system design. We begin by illustrating common features of mathematical optimization problems arising in wireless communication system design. We discuss various scenarios and use cases and their associated mathematical structures from an optimization perspective. We then provide an overview of recently developed optimization techniques in areas ranging from nonconvex optimization, global optimization, and integer programming, to distributed optimization and learning-based optimization. The key to successful solution of mathematical optimization problems is in carefully choosing or developing suitable algorithms (or neural network architectures) that can exploit the underlying problem structure. We conclude the paper by identifying several open research challenges and outlining future research directions. Copyright © 2024, The Authors. All rights reserved.

关键词： Beamforming

TemPrompt: Multi-Task Prompt Learning for Temporal Relation Extraction in RAG-based Crowdsourcing systems

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Yang, Jing Zhao, Yu Yang, Linyao Wang, Xiao Chen, Long Wang, Fei-Yue Institute of Automation Chinese Academy of Sciences Beijing100190 China School of Artificial Intelligence University of Chinese Academy of Sciences Beijing100049 China National Key Laboratory of Information Systems Engineering National University of Defense Technology Changsha410000 China Zhejiang Lab Hangzhou311121 China School of Artificial Intelligence Anhui University Hefei266114 China Department of Computer and Information Science University of Macau 999078 China Faculty of Innovation Engineering Macau University of Science and Technology 999078 China

Temporal relation extraction (TRE) aims to grasp the evolution of events or actions, and thus shape the workflow of associated tasks, so it holds promise in helping understand task requests initiated by requesters in crowdsourcing systems. However, existing methods still struggle with limited and unevenly distributed annotated data. Therefore, inspired by the abundant global knowledge stored within pre-trained language models (PLMs), we propose a multi-task prompt learning framework for TRE (TemPrompt), incorporating prompt tuning and contrastive learning to tackle these issues. To elicit more effective prompts for PLMs, we introduce a task-oriented prompt construction approach that thoroughly takes the myriad factors of TRE into consideration for automatic prompt generation. In addition, we design temporal event reasoning in the form of masked language modeling as auxiliary tasks to bolster the model’s focus on events and temporal cues. The experimental results demonstrate that TemPrompt outperforms all compared baselines across the majority of metrics under both standard and few-shot settings. A case study on designing and manufacturing printed circuit boards is provided to validate its effectiveness in crowdsourcing scenarios. Copyright © 2024, The Authors. All rights reserved.

关键词： Contrastive Learning