检索结果-内蒙古大学图书馆

GradSA: Gradient Sparsification and Accumulation for Communication-Efficient distributed Deep Learning 15th

学校读者我要写书评

暂无评论

GradSA: Gradient Sparsification and Accumulation for Communi...

15th International Conference on Green, Pervasive, and cloud computing, GPC 2020

作者： Liu, Bo Jiang, Wenbin Zhao, Shaofeng Jin, Hai He, Bingsheng National Engineering Research Center for Big Data Technology and System Service Computing Technology and System Lab Cluster and Grid Computing Lab Huazhong University of Science and Technology Wuhan China Wuhan National Laboratory for Optoelectronics Key Laboratory of Information Storage System Engineering Research Center of Data Storage Systems and Technology Huazhong University of Science and Technology Wuhan China Library Henan University of Economics and Law Zhengzhou China Department of Computer Science School of Computing National University of Singapore Singapore Singapore

ISBN: (纸本)9783030642426

Large-scale distributed deep learning is of great importance in various applications. For distributed training, the inter-node gradient communication often becomes the performance bottleneck. Gradient sparsification has been proposed to reduce the communication overhead. However, the sparsification only arbitrarily selects a small fraction of gradients, while the efficiency of selection dimension has been overlooked. Furthermore, gradient staleness is inevitable after applying the sparsification which will ultimately lead to model divergence. In this paper, we propose a staleness-compensated sparse stochastic gradient descent algorithm, GradSA, to improve the training efficiency. Layer-level gradients are sparsified to reduce the communication overhead, which conform to the characteristics of the network structure, and historical accumulation of the approximated gradients is utilized to speed up convergence. We demonstrate the model convergence acceleration and the efficiency of our layer-level selection over existing state-of-the-art works, such as DGC, TernGrad, and 8-Bit quantization. © 2020, Springer Nature Switzerland AG.

关键词： Efficiency

Adaptive Wavelet Filters as Practical Texture Feature Amplifiers for Parkinson’s Disease Screening in OCT

学校读者我要写书评

暂无评论

arXiv 2025年

作者： Zhang, Xiaoqing Shi, Hanfeng Li, Xiangyu Ye, Haili Xu, Tao Li, Na Hu, Yan Lv, Fan Chen, Jiangfan Liu, Jiang Center for High Performance Computing and Shenzhen Key Laboratory of Intelligent Bioinformatics Shenzhen Institute of Advanced Technology Chinese Academy of Sciences Shenzhen China Research Institute of Trustworthy Autonomous Systems Department of Computer Science and Engineering Southern University of Science and Technology Shenzhen518055 China State Key Laboratory of Ophthalmology Optometry and Vision Science Wenzhou Medical University Wenzhou China Centre for Computational Science and Mathematical Modelling Coventry University Coventry United Kingdom School of Computer Science University of Nottingham Ningbo China Ningbo315100 China School of Ophthalmology and Optometry Wenzhou Medical University Wenzhou325035 China Department of Electronic and Information Engineering Changchun University Changchun130022 China

Parkinson’s disease (PD) is a prevalent neurodegenerative disorder globally. The eye’s retina is an extension of the brain and has great potential in PD screening. Recent studies have suggested that texture features extracted from retinal layers can be adopted as biomarkers for PD diagnosis under optical coherence tomography (OCT) images. Frequency domain learning techniques can enhance the feature representations of deep neural networks (DNNs) by decomposing frequency components involving rich texture features. Additionally, previous works have not exploited texture features for automated PD screening in OCT. Motivated by the above analysis, we propose a novel Adaptive Wavelet Filter (AWF) that serves as the Practical Texture Feature Amplifier to fully leverage the merits of texture features to boost the PD screening performance of DNNs with the aid of frequency domain learning. Specifically, AWF first enhances texture feature representation diversities via channel mixer, then emphasizes informative texture feature representations with the well-designed adaptive wavelet filtering token mixer. By combining the AWFs with the DNN stem, AWFNet is constructed for automated PD screening. Additionally, we introduce a novel Balanced Confidence (BC) Loss by mining the potential of sample-wise predicted probabilities of all classes and class frequency prior, to further boost the PD screening performance and trustworthiness of AWFNet. The extensive experiments manifest the superiority of our AWFNet and BC over state-of-the-art methods in terms of PD screening performance and trustworthiness. © 2025, CC BY-NC-SA.

关键词： Neurodegenerative diseases

Guest Editorial: Special Issue on Methods and Infrastructures for Data Mining at the Edge of Internet of Things

学校读者我要写书评

暂无评论

IEEE Internet of Things Journal 2021年第13期8卷 10220-10221页

作者： Fortino, Giancarlo Buyya, Rajkumar Chen, Min Herrera, Francisco Department of Informatics Modeling Electronics and Systems University of Calabria Rende87036 Italy Laboratory School of Computing and Information Systems University of Melbourne MelbourneVIC3010 Australia School of Computer Science and Technology Huazhong University of Science and Technology Wuhan430074 China Computer Science and Artificial Intelligence Department Universidad de Granada Granada18010 Spain

Multimodal Fusion and Vision-Language Models: A Survey for Robot Vision

学校读者我要写书评

暂无评论

arXiv 2025年

作者： Han, Xiaofeng Chen, Shunpeng Fu, Zenghuang Feng, Zhe Fan, Lue An, Dong Wang, Changwei Guo, Li Meng, Weiliang Zhang, Xiaopeng Xu, Rongtao Xu, Shibiao The State Key Laboratory of Multimodal Artificial Intelligence Systems Institute of Automation Chinese Academy of Sciences China School of Artificial Intelligence Beijing University of Posts and Telecommunications China Key Laboratory of Computing Power Network and Information Security Ministry of Education Shandong Computer Science Center China

Robot vision has greatly benefited from advancements in multimodal fusion techniques and vision-language models (VLMs). We systematically review the applications of multimodal fusion in key robotic vision tasks, including semantic scene understanding, simultaneous localization and mapping (SLAM), 3D object detection, navigation and localization, and robot manipulation. We compare VLMs based on large language models (LLMs) with traditional multimodal fusion methods, analyzing their advantages, limitations, and synergies. Additionally, we conduct an in-depth analysis of commonly used datasets, evaluating their applicability and challenges in real-world robotic scenarios. Furthermore, we identify critical research challenges such as cross-modal alignment, efficient fusion strategies, real-time deployment, and domain adaptation, and propose future research directions, including self-supervised learning for robust multimodal representations, transformer-based fusion architectures, and scalable multimodal frameworks. Through a comprehensive review, comparative analysis, and forward-looking discussion, we provide a valuable reference for advancing multimodal perception and interaction in robotic vision. A comprehensive list of studies in this survey is available at https://***/Xiaofeng-Han-Res/MF-RV. Copyright © 2025, The Authors. All rights reserved.

关键词： Supervised learning

Symbiotic evolution of photonics and artificial intelligence: a comprehensive review

学校读者我要写书评

暂无评论

Advanced Photonics 2025年第2期 8-85页

作者： Fu Feng Dewang Huo Ziyang Zhang Yijie Lou Shengyao Wang Zhijuan Gu Dong-Sheng Liu Xinhui Duan Daqian Wang Xiaowei Liu Ji Qi Shaoliang Yu Qingyang Du Guangyong Chen Cuicui Lu Yu Yu Xifeng Ren Xiaocong Yuan Zhejiang Lab Research Center for Frontier Fundamental Studies Westlake Institute for Optoelectronics Zhejiang Key Laboratory of 3D Micro/Nano Fabrication and Characterization Beijing Institute of Technology School of Physics Center for Interdisciplinary Science of Optical Quantum and NEMS Integration Key Laboratory of Advanced Optoelectronic Quantum Architecture and Measurements of Ministry of Education Beijing Key Laboratory of Nanophotonics and Ultrafine Optoelectronic Systems Huazhong University of Science and Technology Wuhan National Laboratory for Optoelectronics University of Science and Technology of China CAS Key Laboratory of Quantum Information University of Science and Technology of China CAS Center for Excellence in Quantum Information and Quantum Physics Zhejiang Lab Research Center for Life Sciences Computing

The rapid advancement of artificial intelligence(AI) has significantly impacted photonics, creating a symbiotic relationship that accelerates the development and applications of both fields. From the perspective of AI aiding photonics, deep-learning methods and various intelligent algorithms have been developed for designing complex photonic structures, where traditional design approaches fall short. AI's capability to process and analyze large data sets has enabled the discovery of novel materials, such as for photovoltaics,leading to enhanced light absorption and efficiency. AI is also significantly transforming the field of optical imaging with improved performance. In addition, AI-driven techniques have revolutionized optical communication systems by optimizing signal processing and enhancing the bandwidth and reliability of data transmission. Conversely, the contribution of photonics to AI is equally profound. Photonic technologies offer unparalleled advantages in the development of AI hardware, providing solutions to overcome the bottlenecks of electronic systems. The implementation of photonic neural networks, leveraging the high speed and parallelism of optical computing, demonstrates significant improvements in the processing speed and energy efficiency of AI computations. Furthermore, advancements in optical sensors and imaging technologies not only enrich AI applications with high-quality data but also expand the capabilities of AI in fields such as autonomous vehicles and medical imaging. We provide comprehensive knowledge and a detailed analysis of the current state of the art, addressing both challenges and opportunities at the intersection of AI and photonics. The multifaceted interactions between AI and photonics will be explored, illustrating how AI has become an indispensable tool in the development of photonics and how photonics, in turn,facilitates advancements in AI. Through a collection of case studies and examples, we underscore the potential

关键词：

Towards Communication-efficient Federated Learning via Sparse and Aligned Adaptive Optimization

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Deng, Xiumei Li, Jun Wei, Kang Shi, Long Xiong, Zeihui Ding, Ming Chen, Wen Jin, Shi Poor, H. Vincent School of Electronic and Optical Engineering Nanjing University of Science and Technology Nanjing210094 China The Department of Computing Hong Kong Polytechnic University 999077 Hong Kong The Pillar of Information Systems Technology and Design Singapore University of Technology and Design Singapore Data61 CSIRO SydneyNSW2015 Australia Department of Electronics Engineering Shanghai Jiao Tong University Shanghai200240 China The National Mobile Communications Research Laboratory Southeast University Nanjing210096 China Department of Electrical and Computer Engineering Princeton University NJ08544 United States

Adaptive moment estimation (Adam), as a Stochastic Gradient Descent (SGD) variant, has gained widespread popularity in federated learning (FL) due to its fast convergence. However, federated Adam (FedAdam) algorithms suffer from a threefold increase in uplink communication overhead compared to federated SGD (FedSGD) algorithms, which arises from the necessity to transmit both local model updates and first and second moment estimates from distributed devices to the centralized server for aggregation. Driven by this issue, we propose a novel sparse FedAdam algorithm called FedAdam-SSM, wherein distributed devices sparsify the updates of local model parameters and moment estimates and subsequently upload the sparse representations to the centralized server. To further reduce the communication overhead, the updates of local model parameters and moment estimates incorporate a shared sparse mask (SSM) into the sparsification process, eliminating the need for three separate sparse masks. Theoretically, we develop an upper bound on the divergence between the local model trained by FedAdam-SSM and the desired model trained by centralized Adam, which is related to sparsification error and imbalanced data distribution. By minimizing the divergence bound between the model trained by FedAdam-SSM and centralized Adam, we optimize the SSM to mitigate the learning performance degradation caused by sparsification error. Additionally, we provide convergence bounds for FedAdam-SSM in both convex and non-convex objective function settings, and investigate the impact of local epoch, learning rate and sparsification ratio on the convergence rate of FedAdam-SSM. Experimental results show that FedAdam-SSM outperforms baselines in terms of convergence rate (over 1.1× faster than the sparse FedAdam baselines) and test accuracy (over 14.5% ahead of the quantized FedAdam baselines). Copyright © 2024, The Authors. All rights reserved.

关键词： Stochastic systems

Spatial computing: Concept, Applications, Challenges and Future Directions

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Yenduri, Gokul Ramalingam, M. Maddikunta, Praveen Kumar Reddy Gadekallu, Thippa Reddy Jhaveri, Rutvij H. Bandi, Ajay Chen, Junxin Wang, Wei Shirawalmath, Adarsh Arunkumar Ravishankar, Raghav Wang, Weizheng School of Computer Science and Engineering VIT-AP University Andhra Pradesh Amaravati522237 India School of Computer Science Engineering and Information Systems Vellore Institute of Technology TamilNadu 632014 India Zhongda Group Haiyan County Zhejiang Province Jiaxing City314312 China College of Information Science and Engineering Jiaxing University Jiaxing314001 China Division of Research and Development Lovely Professional University Phagwara India Department of Computer Science and Engineering School of Technology Pandit Deendayal Energy University Gandhinagar India School of Computer Science and Information Systems Northwest Missouri State University 800 University Dr MaryvilleMO64468 United States School of Software Dalian University of Technology Dalian116621 China Guangdong-Hong Kong-Macao Joint Laboratory for Emotional Intelligence and Pervasive Computing Artificial Intelligence Research Institute Shenzhen MSU-BIT University Guangdong Shenzhen518172 China School of Medical Technology Beijing Institute of Technology Beijing100081 China School of Computer Science and Engineering Vellore Institute of Technology TamilNadu 632014 India Department of Computer Science City University of Hong Kong Hong Kong

Spatial computing is a technological advancement that facilitates the seamless integration of devices into the physical environment, resulting in a more natural and intuitive digital world user experience. Spatial computing has the potential to become a significant advancement in the field of computing. From GPS and location-based services to healthcare, spatial computing technologies have influenced and improved our interactions with the digital world. The use of spatial computing in creating interactive digital environments has become increasingly popular and effective. This is explained by its increasing significance among researchers and industrial organisations, which motivated us to conduct this review. This review provides a detailed overview of spatial computing, including its enabling technologies and its impact on various applications. Projects related to spatial computing are also discussed. In this review, we also explored the potential challenges and limitations of spatial computing. Furthermore, we discuss potential solutions and future directions. Overall, this paper aims to provide a comprehensive understanding of spatial computing, its enabling technologies, their impact on various applications, emerging challenges, and potential solutions. © 2024, CC BY.

关键词： Augmented reality

A Tent Lévy Flying Sparrow Search Algorithm for Feature Selection: A COVID-19 Case Study

学校读者我要写书评

暂无评论

arXiv 2022年

作者： Yang, Qinwen Gao, Yuelin Song, Yanjie School of Computer Science and Engineering North Minzu University Yinchuan750021 China School of Mathematics and Information Science North Minzu University Yinchuan750021 China Ningxia Key Laboratory of Intelligent Information and Big Data Processing Yinchuan750021 China Ningxia Province Cooperative Innovation Center of Scientific Computing and Intelligent in Formation Processing North Minzu University Yinchuan750021 China College of Systems Engineering National University of Defense Technology Changsha410073 China

The "Curse of Dimensionality" induced by the rapid development of information science, might have a negative impact when dealing with big datasets. In this paper, we propose a variant of the sparrow search algorithm (SSA), called Tent Lévy flying sparrow search algorithm (TFSSA), and use it to select the best subset of features in the packing pattern for classification purposes. SSA is a recently proposed algorithm that has not been systematically applied to feature selection problems. After verification by the CEC2020 benchmark function, TFSSA is used to select the best feature combination to maximize classification accuracy and minimize the number of selected features. The proposed TFSSA is compared with nine algorithms in the literature. Nine evaluation metrics are used to properly evaluate and compare the performance of these algorithms on twenty-one datasets from the UCI repository. Furthermore, the approach is applied to the coronavirus disease (COVID-19) dataset, yielding the best average classification accuracy and the average number of feature selections, respectively, of 93.47% and 2.1. Experimental results confirm the advantages of the proposed algorithm in improving classification accuracy and reducing the number of selected features compared to other wrapper-based algorithms. Copyright © 2022, The Authors. All rights reserved.

关键词： COVID-19

distributed Pinning Set Stabilization of Large-Scale Boolean Networks

学校读者我要写书评

暂无评论

arXiv 2022年

作者： Zhu, Shiyong Lu, Jianquan Sun, Liangjie Cao, Jinde The Department of Systems Science School of Mathematics Southeast University Nanjing210096 China The Advanced Modeling and Applied Computing Laboratory Department of Mathematics The University of Hong Kong Hong Kong The School of Mathematics Frontiers Science Center for Mobile Information Communication and Security Southeast University Nanjing210096 China The Purple Mountain Laboratories Nanjing211111 China The Yonsei Frontier Lab Yonsei University Seoul03722 Korea Republic of

In this article, we design the distributed pinning controllers to globally stabilize a Boolean network (BN), specially a sparsely connected large-scale one, towards a preassigned subset of state space through the node-to-node message exchange. Given an appointed state set, system nodes are partitioned into two disjoint parts, which respectively gather the nodes whose states are fixed or arbitrary with respect to the given state set. With such node division, three parts of pinned nodes are selected and the state feedback controllers are accordingly designed such that the resulting BN satisfies three conditions: the states of the other nodes cannot affect the nodal dynamics of fixed-state nodes, the subgraph of network structure induced by the fixed-state nodes is acyclic, and the steady state of the subnetwork induced by the fixed-state nodes lies in the state set given beforehand. If the BN after control is acyclic, the stabilizing time is revealed to be no more than the length of the longest path in the current network structure plus one. This enables us to further design the pinning controllers with the constraint of stabilizing time. Noting that the overall procedure runs in an exponentially increasing time with respect to the largest number of functional variables in the dynamics of pinned nodes, the sparsely-connected large-scale BNs can be well addressed in a reasonable amount of time. Finally, we demonstrate the applications of our theoretical results in a T-LGL survival signal network with 29 nodes and T-cell receptor signaling network with 90 nodes. Copyright © 2022, The Authors. All rights reserved.

关键词： Controllers