检索结果-内蒙古大学图书馆

Federated Cubic Regularized Newton Learning with Sparsification-amplified Differential Privacy

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Huo, Wei Liu, Changxin Ding, Kemi Johansson, Karl Henrik Shi, Ling Department of Electronic and Computer Engineering Hong Kong University of Science and Technology Hong Kong Division of Decision and Control Systems School of Electrical Engineering and Computer Science KTH Royal Institute of Technology Sweden Digital Futures StockholmSE-10044 Sweden School of System Design and Intelligent Manufacturing Southern University of Science and Technology Shenzhen518055 China

This paper investigates the use of the cubic-regularized Newton method within a federated learning framework while addressing two major concerns that commonly arise in federated learning: privacy leakage and communication bottleneck. We introduce a federated learning algorithm called Differentially Private Federated Cubic Regularized Newton (DP-FCRN). By leveraging second-order techniques, our algorithm achieves lower iteration complexity compared to first-order methods. We also incorporate noise perturbation during local computations to ensure privacy. Furthermore, we employ sparsification in uplink transmission, which not only reduces the communication costs but also amplifies the privacy guarantee. Specifically, this approach reduces the necessary noise intensity without compromising privacy protection. We analyze the convergence properties of our algorithm and establish the privacy guarantee. Finally, we validate the effectiveness of the proposed algorithm through experiments on a benchmark dataset. © 2024, CC BY.

关键词： Federated learning

Deep learning model compression using network sensitivity and gradients

学校读者我要写书评

暂无评论

arXiv 2022年

作者： Sakthi, Madhumitha Yadla, Niranjan Pawate, Raj Electrical and Computer Engineering The University of Texas Austin United States Tensilica IPG Cadence Design Systems Inc. United States

Deep learning model compression is an improving and important field for the edge deployment of deep learning models. Given the increasing size of the models and their corresponding power consumption, it is vital to decrease the model size and compute requirement without a significant drop in the model’s performance. In this paper, we present model compression algorithms for both non-retraining and retraining conditions. In the first case where retraining of the model is not feasible due to lack of access to the original data or absence of necessary compute resources while only having access to off-the-shelf models, we propose the Bin & Quant algorithm for compression of the deep learning models using the sensitivity of the network parameters. This results in 13x compression of the speech command and control model and 7x compression of the DeepSpeech2 models. In the second case when the models can be retrained and utmost compression is required for the negligible loss in accuracy, we propose our novel gradient-weighted k-means clustering algorithm (GWK). This method uses the gradients in identifying the important weight values in a given cluster and nudges the centroid towards those values, thereby giving importance to sensitive weights. Our method effectively combines product quantization with the EWGS [1] algorithm for sub-1bit representation of the quantized models. We test our GWK algorithm on the CIFAR10 dataset across a range of models such as ResNet20, ResNet56, MobileNetv2 and show 35x compression on quantized models for less than 2% absolute loss in accuracy compared to the floating-point models. © 2022, CC BY.

关键词： Image classification

Deep Learning-Based Automated Segmentation of Fiber Breaks in Advanced Composites 38

学校读者我要写书评

暂无评论

Deep Learning-Based Automated Segmentation of Fiber Breaks i...

38th Technical Conference of the American Society for Composites, ASC 2023

作者： Jain, Sudhir Li, Carina Xiaochen Vuong, Daniel McGllick, Matthew Saifaee, Saeed Furtado, Carolina Kopp, Reed Joseph, Joshua Adjodah, Dhaval Rosini, Sebastian Sinclair, Ian Spearing, S. Mark Wardle, Brian L. Integrated Design and Management Program Massachusetts Institute of Technology CambridgeMA02139 United States Department of Aeronautics and Astronautics Massachusetts Institute of Technology CambridgeMA02139 United States Department of Electrical Engineering and Computer Science Massachusetts Institute of Technology CambridgeMA02139 United States MIT Media Lab Massachusetts Institute of Technology CambridgeMA02139 United States Department of Mechanical Engineering University of Southampton SouthamptonSO17 1BJ United Kingdom Departments of Aeronautics and Astronautics and Mechanical Engineering Massachusetts Institute of Technology CambridgeMA02139 United States

ISBN: (纸本)9781605956916

The heterogeneity of the microstructure and the anisotropy of the mechanical properties of carbon fiber reinforced polymer composites (CFRP) are associated with complex damage mechanisms that make failure prediction highly challenging. Typical damage mechanisms for CFRP include: (i) interlaminar cracking (delamination), (ii) matrix cracking, and (iii) fiber breakage. X-ray computed tomography (CT) has been utilized recently to identify the complex damage in progressive failure of CFRP in 3D and 4D (3D spatial and 1D temporal). However, these scans contain a large amount of data (≈10 GB/mm3), which makes it challenging to extract mechanistic insights in a reasonable time frame. In previous work, we presented the first deep learning (DL) convolutional neural network (CNN) segmentation of multiclass microscale matrix damage in CFRP. The trained CNN segments complex and sparse composite matrix damage mechanism to ≈ 99.99% agreement, with nearly 100% of human time eliminated, providing higher accuracy compared to human segmentation. In this work, we developed CNN-based method for the automated segmentation of fiber breaks using synchrotron radiation-based CT (SRCT) scans that have been human-labeled. The model employed a U-Net full CNN as the high-level encoder (feature extraction)-decoder (feature localization) structure, and VGG16 as the deep CNN backbone via the publicly available Segmentation Models GitHub repository. An unbiased evaluation was performed comparing the machine vs. the human-labeled ground truth. Finally, active learning was presented for the machine-assisted relabeling of datasets containing fiber breaks. The deep learning-based automated segmentation method demonstrated accurate identification (> 90%) of fiber breakage for CFRP. © 2023 by DEStech Publications, Inc. and American Society for Composites. All rights reserved.

关键词： Complex networks

Generative AI-empowered Simulation for Autonomous Driving in Vehicular Mixed Reality Metaverses

学校读者我要写书评

暂无评论

arXiv 2023年

作者： Xu, Minrui Niyato, Dusit Chen, Junlong Zhang, Hongliang Kang, Jiawen Xiong, Zehui Mao, Shiwen Han, Zhu The School of Computer Science and Engineering Nanyang Technological University Singapore308232 Singapore The School of Electronics Peking University Beijing100871 China The School of Automation Guangdong University of Technology China The Pillar of Information Systems Technology and Design Singapore University of Technology and Design Singapore487372 Singapore The Department of Electrical and Computer Engineering Auburn University AuburnAL36849-5201 United States The Department of Electrical and Computer Engineering University of Houston HoustonTX77004 United States The Department of Computer Science and Engineering Kyung Hee University Seoul446-701 Korea Republic of

In the vehicular mixed reality (MR) Metaverse, the distance between physical and virtual entities can be overcome by fusing the physical and virtual environments with multidimensional communications in autonomous driving systems. Assisted by digital twin (DT) technologies, connected autonomous vehicles (AVs), roadside units (RSU), and virtual simulators can maintain the vehicular MR Metaverse via digital simulations for sharing data and making driving decisions collaboratively. However, large-scale traffic and driving simulation via realistic data collection and fusion from the physical world for online prediction and offline training in autonomous driving systems are difficult and costly. In this paper, we propose an autonomous driving architecture, where generative AI is leveraged to synthesize unlimited conditioned traffic and driving data in simulations for improving driving safety and traffic efficiency. First, we propose a multi-task DT offloading model for the reliable execution of heterogeneous DT tasks with different requirements at RSUs. Then, based on the preferences of AV’s DTs and collected realistic data, virtual simulators can synthesize unlimited conditioned driving and traffic datasets to further improve robustness. Finally, we propose a multi-task enhanced auction-based mechanism to provide fine-grained incentives for RSUs in providing resources for autonomous driving. The property analysis and experimental results demonstrate that the proposed mechanism and architecture are strategy-proof and effective, respectively. © 2023, CC BY.

关键词： Autonomous vehicles

Sparks of GPTs in Edge Intelligence for Metaverse: Caching and Inference for Mobile AIGC Services

学校读者我要写书评

暂无评论

arXiv 2023年

作者： Xu, Minrui Niyato, Dusit Zhang, Hongliang Kang, Jiawen Xiong, Zehui Mao, Shiwen Han, Zhu School of Computer Science and Engineering Nanyang Technological University Singapore639798 Singapore School of Electronics Peking University Beijing100871 China School of Automation Guangdong University of Technology China Pillar of Information Systems Technology and Design Singapore University of Technology and Design Singapore487372 Singapore Department of Electrical and Computer Engineering Auburn University AuburnAL36849-5201 United States Department of Electrical and Computer Engineering University of Houston HoustonTX77004 United States Department of Computer Science and Engineering Kyung Hee University Seoul446-701 Korea Republic of

Aiming at achieving artificial general intelligence (AGI) for Metaverse, pretrained foundation models (PFMs), e.g., generative pretrained transformers (GPTs), can effectively provide various AI services, such as autonomous driving, digital twins, and AI-generated content (AIGC) for extended reality. With the advantages of low latency and privacy-preserving, serving PFMs of mobile AI services in edge intelligence is a viable solution for caching and executing PFMs on edge servers with limited computing resources and GPU memory. However, PFMs typically consist of billions of parameters that are computation and memory-intensive for edge servers during loading and execution. In this article, we investigate edge PFM serving problems for mobile AIGC services of Metaverse. First, we introduce the fundamentals of PFMs and discuss their characteristic fine-tuning and inference methods in edge intelligence. Then, we propose a novel framework of joint model caching and inference for managing models and allocating resources to satisfy users’ requests efficiently. Furthermore, considering the in-context learning ability of PFMs, we propose a new metric to evaluate the freshness and relevance between examples in demonstrations and executing tasks, namely the Age of Context (AoC). Finally, we propose a least context algorithm for managing cached models at edge servers by balancing the tradeoff among latency, energy consumption, and accuracy. © 2023, CC BY.

关键词： Energy utilization

Generative AI-empowered Effective Physical-Virtual Synchronization in the Vehicular Metaverse

学校读者我要写书评

暂无评论

arXiv 2023年

作者： Xu, Minrui Niyato, Dusit Zhang, Hongliang Kang, Jiawen Xiong, Zehui Mao, Shiwen Han, Zhu The School of Computer Science and Engineering Nanyang Technological University Singapore308232 Singapore The School of Electronics Peking University Beijing100871 China The School of Automation Guangdong University of Technology China The Pillar of Information Systems Technology and Design Singapore University of Technology and Design Singapore487372 Singapore The Department of Electrical and Computer Engineering Auburn University AuburnAL36849-5201 United States The Department of Electrical and Computer Engineering University of Houston HoustonTX77004 United States The Department of Computer Science and Engineering Kyung Hee University Seoul446-701 Korea Republic of

Metaverse seamlessly blends the physical world and virtual space via ubiquitous communication and computing infrastructure. In transportation systems, the vehicular Metaverse can provide a fully-immersive and hyperreal traveling experience (e.g., via augmented reality head-up displays, AR-HUDs) to drivers and users in autonomous vehicles (AVs) via roadside units (RSUs). However, provisioning real-time and immersive services necessitates effective physical-virtual synchronization between physical and virtual entities, i.e., AVs and Metaverse AR recommenders (MARs). In this paper, we propose a generative AI-empowered physical-virtual synchronization framework for the vehicular Metaverse. In physical-to-virtual synchronization, digital twin (DT) tasks generated by AVs are offloaded for execution in RSU with future route generation. In virtual-to-physical synchronization, MARs customize diverse and personal AR recommendations via generative AI models based on user preferences. Furthermore, we propose a multi-task enhanced auction-based mechanism to match and price AVs and MARs for RSUs to provision real-time and effective services. Finally, property analysis and experimental results demonstrate that the proposed mechanism is strategy-proof and adverse-selection free while increasing social surplus by 50%. © 2023, CC BY.

关键词： Augmented reality

DeepOHeat-v1: Efficient Operator Learning for Fast and Trustworthy Thermal Simulation and Optimization in 3D-IC design

学校读者我要写书评

暂无评论

arXiv 2025年

作者： Yu, Xinling Liu, Ziyue Li, Hai Li, Yixing Ai, Xin Zeng, Zhiyu Young, Ian Zhang, Zheng Department of Electrical and Computer Engineering University of California Santa BarbaraCA93106 United States Department of Computer Science University of California Santa BarbaraCA93106 United States Cadence Design Systems AustinTX78759 United States Intel Corporation HillsboroOR97124 United States

Thermal analysis is crucial in three-dimensional integrated circuit (3D-IC) design due to increased power density and complex heat dissipation paths. Although operator learning frameworks such as DeepOHeat [1] have demonstrated promising preliminary results in accelerating thermal simulation, they face critical limitations in prediction capability for multi-scale thermal patterns, training efficiency, and trustworthiness of results during design optimization. This paper presents DeepOHeat-v1, an enhanced physics-informed operator learning framework that addresses these challenges through three key innovations. First, we integrate Kolmogorov-Arnold Networks with learnable activation functions as trunk networks, enabling an adaptive representation of multi-scale thermal patterns. This approach achieves a 1.25× and 6.29× reduction in error in two representative test cases. Second, we introduce a separable training method that decomposes the basis function along the coordinate axes, achieving 62× training speedup and 31× GPU memory reduction in our baseline case, and enabling thermal analysis at resolutions previously infeasible due to GPU memory constraints. Third, we propose a confidence score to evaluate the trustworthiness of the predicted results, and further develop a hybrid optimization workflow that combines operator learning with finite difference (FD) using Generalized Minimal Residual (GMRES) method for incremental solution refinement, enabling efficient and trustworthy thermal optimization. Experimental results demonstrate that DeepOHeatv1 achieves accuracy comparable to optimization using high-fidelity finite difference solvers, while speeding up the entire optimization process by 70.6× in our test cases, effectively minimizing the peak temperature through optimal placement of heat-generating components. © 2025, CC BY-NC-ND.

关键词： Integrated circuit layout

Estimating Higher-Order Mixed Memberships via the 2,∞ Tensor Perturbation Bound

学校读者我要写书评

暂无评论

arXiv 2022年

作者： Agterberg, Joshua Zhang, Anru R. Departments of Electrical and Systems Engineering and Statistics and Data Science University of Pennsylvania United States Departments of Biostatistics & Bioinformatics Computer Science Mathematics and Statistical Science Duke University United States

Higher-order multiway data is ubiquitous in machine learning and statistics and often exhibits community-like structures, where each component (node) along each different mode has a community membership associated with it. In this paper we propose the tensor mixed-membership blockmodel, a generalization of the tensor blockmodel positing that memberships need not be discrete, but instead are convex combinations of latent communities. We establish the identifiability of our model and propose a computationally efficient estimation procedure based on the higher-order orthogonal iteration algorithm (HOOI) for tensor SVD composed with a simplex corner-finding algorithm. We then demonstrate the consistency of our estimation procedure by providing a per-node error bound, which showcases the effect of higher-order structures on estimation accuracy. To prove our consistency result, we develop the 2,∞ tensor perturbation bound for HOOI under independent, heteroskedastic, subgaussian noise that may be of independent interest. Our analysis uses a novel leave-one-out construction for the iterates, and our bounds depend only on spectral properties of the underlying low-rank tensor under nearly optimal signal-to-noise ratio conditions such that tensor SVD is computationally feasible. Finally, we apply our methodology to real and simulated data, demonstrating some effects not identifiable from the model with discrete community memberships. © 2022, CC BY.

关键词： Tensors

QoE-Guaranteed Optimization in MEC-Enabled Metaverse: An Active Inference Deep Reinforcement Learning Approach

学校读者我要写书评

暂无评论

IEEE Transactions on Cognitive Communications and Networking 2025年

作者： Du, Jianbo Gong, Jie Chu, Xiaoli Xiong, Zehui Chen, Xianfu Dong, Mianxiong Yu, F. Richard Xi'an University of Posts and Telecommunications Shaanxi Key Laboratory of Information Communication Network and Security School of Communications and Information Engineering Xi'an710121 China The University of Sheffield Mappin Street Department of Electronic and Electrical Engineering SheffieldS1 3JD United Kingdom Singapore University of Technology and Design Information Systems Technology and Design Pillar Singapore Singapore Shenzhen CyberAray Network Technology Company Ltd Shenzhen518000 China Muroran Institute of Technology Department of Sciences and Informatics Muroran050-8585 Japan Carleton University Systems and Computer Engineering OttawaON Canada

In this paper, we consider a MEC-enabled metaverse scenario which consists of a remote metaverse server and an edge server that cooperates to provide services to mobile users. The edge server is deployed at the base station (BS), serves a dual role: augmenting computational capabilities for user equipment (UE) and pre-caching a portion of the metaverse service contents before each time slot. Moreover, the foreground information and the requested contents generated by the UEs can also be cached to the BS. We formulate a problem to maximize the cache hit number by jointly optimizing contents pre-caching and resource allocation at the BS while considering UEs preference and reducing the UEs total energy consumption, essential for the efficient delivery of services in dynamic MEC environments. To solve this problem, we reformulate it as a partially observable markov decision process and propose an active inference enabled deep reinforcement learning algorithm, which combines active inference with deep reinforcement learning to select the optimal strategy by minimizing the expected free energy. Simulations show that the proposed algorithm can effectively improve the total quality of experience and the cache hit number of UEs, while minimizing the UEs total energy consumption compared with other baseline algorithms © 2015 IEEE.

关键词： Active learning