检索结果-内蒙古大学图书馆

Communication-Efficient Personalized Distributed Learning with Data and Node Heterogeneity

IEEE Transactions on Cognitive Communications and Networking 2025年

作者： Tian, Zhuojun Zhang, Zhaoyang Li, Yiwei Bennis, Mehdi Zhejiang University College of Information Science and Electronic Engineering Hangzhou310027 China University of Oulu Center for Wireless Communications Oulu90014 Finland State Key Laboratory of Industrial Control Technology Hangzhou310027 China Zhejiang Provincial Key Laboratory of Multimodal Communication Networks and Intelligent Information Processing Hangzhou310027 China Xiamen University of Technology Fujian Key Laboratory of Communication Network and Information Processing Xiamen361024 China

To jointly tackle the challenges of data and node heterogeneity in decentralized learning, we propose a distributed strong lottery ticket hypothesis (DSLTH), based on which a communication-efficient personalized learning algorithm is developed. In the proposed method, each local model is represented as the ‌Hadamard product of global real-valued parameters and a personalized binary mask for pruning. The local model is learned by updating and fusing the personalized binary masks while the real-valued parameters are fixed among different agents. To further reduce the complexity of hardware implementation, we incorporate a group sparse regularization term in the loss function, enabling the learned local model to achieve structured sparsity. Then, a binary mask aggregation algorithm is designed by introducing an intermediate aggregation tensor and adding a personalized fine-tuning step in each iteration, which constrains model updates towards the local data distribution. The proposed method effectively leverages the relativity among agents while meeting personalized requirements in heterogeneous node conditions. We also provide a theoretical proof for the DSLTH, establishing it as the foundation of the proposed method. Numerical simulations confirm the validity of the DSLTH and demonstrate the effectiveness of the proposed algorithm. © 2015 IEEE.

关键词： Iterative methods

来源：评论

学校读者我要写书评

暂无评论

A fault isolation method using kernel canonical variate analysis-based contribution

A fault isolation method using kernel canonical variate anal...

引用

第35届中国过程控制会议

作者： Hongxin Wang Lingxia Mu Nan Feng Han Liu Ding Liu Department of Information and Control Xi'an University of Technology Shaanxi Key Laboratory of Complex System Control and Intelligent Information Processing Xi'an University of Technology School of Intelligence Science and Technology University of Science and Technology Beijing

Fault diagnosis and isolation is important for industrial system. In this paper, a kernel canonical variate analysis(KCVA) is proposed for fault isolation. KCVA is originally used as a data dimension reduction technique which can account for nonlinearity and correlations in the industrial dynamical process data. But there are some difficulties using KCVA in the construction of the contribution for the fault isolation. On the one hand, it is difficult to compute the contributions of individual variables because it is scarcely possible to find an inverse mapping from the feature space to the original space. On the other hand, a smearing effect is hardly avoided. To solve the problem, a KCVA-based contributions is proposed using the state subspace and the residual subspace which can isolate the faulty variables effectively. Simulations are conducted on the Tennessee Eastman process to verify the performance of the proposed method.

关键词： Kernel canonical variate analysis contribution plot fault isolation

来源：评论

学校读者我要写书评

暂无评论

NPGA: A Unified Algorithmic Framework for Decentralized Constraint-Coupled Optimization

arXiv

引用

arXiv 2022年

作者： Li, Jingwang Su, Housheng School of Artificial Intelligence and Automation The Key Laboratory of Image Processing and Intelligent Control of Education Ministry of China Huazhong University of Science and Technology Wuhan430074 China

This work focuses on a class of general decentralized constraint-coupled optimization problems. We propose a novel nested primal-dual gradient algorithm (NPGA), which can achieve linear convergence under the weakest known condition, and its theoretical convergence rate surpasses all known results. More importantly, NPGA serves not only as an algorithm but also as a unified algorithmic framework, encompassing various existing algorithms as special cases. By designing different network matrices, we can derive numerous versions of NPGA and analyze their convergences by leveraging the convergence results of NPGA conveniently, thereby enabling the design of more efficient algorithms. Finally, we conduct numerical experiments to compare the convergence rates of NPGA and existing algorithms, providing empirical evidence for the superior performance of NPGA. Copyright © 2022, The Authors. All rights reserved.

关键词： Consensus algorithm

来源：评论

学校读者我要写书评

暂无评论

An Amalgamated CNN-Transformer Network for Lightweight image Super-Resolution

Journal of Network Intelligence

引用

Journal of Network Intelligence 2024年第3期9卷 1376-1387页

作者： Fang, Jinsheng Lin, Hanjiang Zeng, Kun School of Computer Science and Engineering Minnan Normal University Zhangzhou363000 China Key Laboratory of Data Science and Intelligence Application Fujian Province University Fujian Zhangzhou363000 China School of Computer and Control engineering Fujian Provincial Key Laboratory of Information Processing and Intelligent Control Minjiang University Fuzhou350108 China

Recently, Transformer-based methods for single image super-resolution (SISR) have achieved better performance advantages than the methods based on convolutional neural network (CNN). Exploiting self-attention mechanism to model global context definitely improves the SR results. However, the neglect of local information will bring inevitable reduction of the network performance. In this work, we propose an Amalgamated CNN-Transformer network for lightweight SR, namely ACTSR. Specifically, an amalgamated CNN-Transformer block (ACTB) is developed to extract the useful information of both local and global features. By employing stacked ACTBs, our ACTSR extracts more informative deep features beneficially for super-resolution reconstruction to improve network performance while keeps lightweight and flexible enough. Extensive experiments on commonly used benchmark datasets validate our ACTSR outperforms the advanced competitors. Our codes are available at: https://***/ginsengf/ACTSR. © 2024, Taiwan Ubiquitous Information CO LTD. All rights reserved.

关键词： Convolutional neural networks

来源：评论

学校读者我要写书评

暂无评论

GraphCFC: A Directed Graph Based Cross-Modal Feature Complementation Approach for Multimodal Conversational Emotion Recognition

arXiv

引用

arXiv 2022年

作者： Li, Jiang Wang, Xiaoping Lv, Guoqing Zeng, Zhigang The School of Artificial Intelligence and Automation The Key Laboratory of Image Processing and Intelligent Control of Education Ministry of China Huazhong University of Science and Technology Wuhan430074 China

Emotion Recognition in Conversation (ERC) plays a significant part in Human-Computer Interaction (HCI) systems since it can provide empathetic services. Multimodal ERC can mitigate the drawbacks of uni-modal approaches. Recently, Graph Neural Networks (GNNs) have been widely used in a variety of fields due to their superior performance in relation modeling. In multimodal ERC, GNNs are capable of extracting both long-distance contextual information and inter-modal interactive information. Unfortunately, since existing methods such as MMGCN directly fuse multiple modalities, redundant information may be generated and diverse information may be lost. In this work, we present a directed Graph based Cross-modal Feature Complementation (GraphCFC) module that can efficiently model contextual and interactive information. GraphCFC alleviates the problem of heterogeneity gap in multimodal fusion by utilizing multiple subspace extractors and Pair-wise Cross-modal Complementary (PairCC) strategy. We extract various types of edges from the constructed graph for encoding, thus enabling GNNs to extract crucial contextual and interactive information more accurately when performing message passing. Furthermore, we design a GNN structure called GAT-MLP, which can provide a new unified network framework for multimodal learning. The experimental results on two benchmark datasets show that our GraphCFC outperforms the state-of-the-art (SOTA) approaches. Copyright © 2022, The Authors. All rights reserved.

关键词： Emotion Recognition

来源：评论

学校读者我要写书评

暂无评论

Implicit Tracking-Based Distributed Constraint-Coupled Optimization

arXiv

引用

arXiv 2022年

作者： Li, Jingwang Su, Housheng The School of Artificial Intelligence and Automation The Key Laboratory of Image Processing and Intelligent Control of Education Ministry of China Huazhong University of Science and Technology Wuhan430074 China

A class of distributed optimization problem with a globally coupled equality constraint and local constrained sets is studied in this paper. For its special case where local constrained sets are absent, an augmented primal-dual gradient dynamics is proposed and analyzed, but it cannot be implemented distributedly since the violation of the coupled constraint needs to be used. Benefiting from the brand-new comprehending of a classical distributed unconstrained optimization algorithm, the novel implicit tracking approach is proposed to track the violation distributedly, which leads to the birth of the implicit tracking-based distributed augmented primal-dual gradient dynamics (IDEA). A projected variant of IDEA, i.e., Proj-IDEA, is further designed to deal with the general case where local constrained sets exist. With the aid of the Lyapunov stability theory, the convergences of IDEA and Proj-IDEA over undigraphs and digraphs are analyzed respectively. As far as we know, Proj-IDEA is the first constant step-size distributed algorithm which can solve the studied problem without the need of the strict convexity of local cost functions. Besides, if local cost functions are strongly convex and smooth, IDEA can achieve exponential convergence with a weaker condition about the coupled constraint. Finally, numerical experiments are taken to corroborate our theoretical results. © 2022, CC BY.

关键词： Cost functions

来源：评论

学校读者我要写书评

暂无评论

Short-term Traffic Flow Prediction by Graph Deep Learning with Spatial Temporal Modeling 2

Short-term Traffic Flow Prediction by Graph Deep Learning wi...

引用

2nd International Conference on Electronic Information Technology and Smart Agriculture, ICEITSA 2022

作者： Yuan, Lilian Fang, Weilin Xiao, Hongbo Xiao, Jianhua Shi, Yuanquan Yang, Yujun School of Computer Science and Engineering Huaihua University Hunan Huaihua418000 China Intelligent Processing and Application in Hunan Province Universities Key Laboratory of Wuling-Mountain Health Big Data Hunan Huaihua418000 China Wuling-Mountain Ecological Agriculture in Hunan Province Key Laboratory of Intelligent Control Technology for Huaihua Hunan418000 China

ISBN: (纸本)9781665464017

An extensive amount of research applications on traffic flow prediction models taking into account spatio-temporal modeling have been undertaken in recent years. Relative to traditional spatio-temporal modeling approaches, good application results have been achieved in predicting traffic flow. However, in real life, traffic flow proves to be time- and space-dependent, which poses difficulties for accurate modeling of traffic flow prediction. To address these problems, we propose the ST-GNN-GRU model, which mainly contains S-GNN and T-GRU modules. The model uses a combination of spatio-temporal modeling and deep learning, which can not only cope with the spatio-temporal modeling problem, but also improve the traffic flow prediction accuracy. The experiments are conducted using the METR-LA dataset, and the outcomes show that the proposed method performs more effectively than other baseline approaches. © 2022 IEEE.

关键词： Graph neural networks

来源：评论

学校读者我要写书评

暂无评论

Lithium-ion battery state-of-charge estimation using a real-time moving horizon estimation algorithm

Lithium-ion battery state-of-charge estimation using a real-...

引用

2021 CAA Symposium on Fault Detection, Supervision, and Safety for Technical Processes, SAFEPROCESS 2021

作者： Yan, Jiayu Li, Song Wan, Yiming Huazhong University of Science and Technology Key Laboratory of Image Processing and Intelligent Control Ministry of Education Wuhan China 29th Research Institute of China Electronics Technology Group Corporation Chengdu China

ISBN: (纸本)9781665401159

For battery management systems, it is significant to reliably estimate state-of-charge (SOC) from limited measurements in real time. Based on a nonlinear SOC-dependent equivalent circuit model, we propose a real-time moving horizon estimation (MHE) algorithm for SOC estimation. For efficient computation, the sequential quadratic programming strategy is applied, and each quadratic programming (QP) iteration is solved via the structure-exploiting forward and backward recursions. To account for the limited computation time allowed within each sampling interval, the proposed algorithm does not fully solve the receding horizon optimization problem over each sliding window, but performs fixed number of QP iterations. The simulation results show that compared to the existing fully solved MHE applied to SOC estimation, the proposed real-time MHE algorithm significantly saves computation time at the cost of a slight loss of estimation performance. © 2021 IEEE.

关键词： Quadratic programming

来源：评论

学校读者我要写书评

暂无评论

Multimodal Representations Learning Based on Mutual Information Maximization and Minimization and Identity Embedding for Multimodal Sentiment Analysis

arXiv

引用

arXiv 2022年

作者： Zheng, Jiahao Zhang, Sen Wang, Xiaoping Zeng, Zhigang School of Artificial Intelligence and Automation The Key Laboratory of Image Processing Intelligent Control of Education Ministry of China Huazhong University of Science and Technology Wuhan430074 China

Multimodal sentiment analysis (MSA) is a fundamental complex research problem due to the heterogeneity gap between different modalities and the ambiguity of human emotional expression. Although there have been many successful attempts to construct multimodal representations for MSA, there are still two challenges to be addressed: 1) A more robust multimodal representation needs to be constructed to bridge the heterogeneity gap and cope with the complex multimodal interactions, and 2) the contextual dynamics must be modeled effectively throughout the information flow. In this work, we propose a multimodal representation model based on Mutual information Maximization and Minimization and Identity Embedding (MMMIE). We combine mutual information maximization between modal pairs, and mutual information minimization between input data and corresponding features to mine the modal-invariant and task-related information. Furthermore, Identity Embedding is proposed to prompt the downstream network to perceive the contextual information. Experimental results on two public datasets demonstrate the effectiveness of the proposed model. Copyright © 2022, The Authors. All rights reserved.

关键词： Sentiment analysis

来源：评论

学校读者我要写书评

暂无评论

An Adaptive Nonlinear Filter for Uncertain Measurement Noise

An Adaptive Nonlinear Filter for Uncertain Measurement Noise

引用

International Conference on Guidance, Navigation and control, ICGNC 2020

作者： Dong, Xiangxiang Zhang, Lulu Lv, Runyan Cai, Yunze Department of Automation Shanghai Jiao Tong University Shanghai200240 China Key Laboratory of System Control and Information Processing Shanghai Jiao Tong University Shanghai200240 China Key Laboratory of Marine Intelligent Equipment and System Shanghai Jiao Tong University Shanghai200240 China

ISBN: (纸本)9789811581540

An improved adaptive variational Bayesian (VB) nonlinear filter is proposed. Through Cubature sampling, inverse Wishart distribution is introduced to jointly estimate system states and inaccurate measurement noise for nonlinear systems. Furthermore, to avoid non-positive definite problem of the state covariance matrix in joint estimation, Singular Value Decomposition (SVD) is introduced to VBCKF, based on which SVD-VBCKF is designed. Besides, computational complexity of proposed filter is analyzed theoretically. Simulation results demonstrate that proposed SVD-VBCKF has higher filtering accuracy. © 2022, The Editor(s) (if applicable) and The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd.

关键词： Singular value decomposition

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：