检索结果-内蒙古大学图书馆

Learning dynamics of gradient descent optimization in deep neural networks

science China(Information sciences) 2021年第5期64卷 17-31页

作者： Wei WU Xiaoyuan JING Wencai DU Guoliang CHEN School of Computer Science Wuhan University Institute of Data Science City University of Macau College of Computer Science and Software Engineering Shenzhen University

Stochastic gradient descent(SGD)-based optimizers play a key role in most deep learning models,yet the learning dynamics of the complex model remain obscure. SGD is the basic tool to optimize model parameters, and is improved in many derived forms including SGD momentum and Nesterov accelerated gradient(NAG). However, the learning dynamics of optimizer parameters have seldom been studied. We propose to understand the model dynamics from the perspective of control theory. We use the status transfer function to approximate parameter dynamics for different optimizers as the first-or second-order control system, thus explaining how the parameters theoretically affect the stability and convergence time of deep learning models, and verify our findings by numerical experiments.

关键词： learning dynamics deep neural networks gradient descent control model transfer function

来源：评论

学校读者我要写书评

暂无评论

Multi-Server Cooperative Video Caching Strategy Based on Deep Reinforcement Learning in Cloud-Edge Computing

Multi-Server Cooperative Video Caching Strategy Based on Dee...

引用

International Conference on big data Computing and Communications (bigCOM)

作者： Guanyu Li Yingchi Mao XinXin Peng Haotian Zheng Zibo Wang College of Computer Science and Software Engineering Hohai University Nanjing China Key Laboratory of Water Big Data Technology of Ministry of Water Resources Hohai University Nanjing China Huaneng Hydro Lancang Xiaowan Hydropower Plant Dali China

ISBN: (数字)9798331509538

ISBN: (纸本)9798331509545

The soaring number of mobile devices has led the innovation of the edge caching in edge computing, which relieves the heavy pressure of cloud computing. However, with the escalating popularity of short video services, the prevalence of video content in edge server caches has seen a marked increase. Previous work ignored the impact of edge server performance on video caching and failed to automatically adjust the optimal video bit rate in response to users' demand. To improve users' evaluation for watching videos on devices with different display sizes, we propose a novel edge caching strategy called Gate Self-attention Recurrent Network-Multi Agent Reinforcement Learning(GSRN-MARL). GSRN-MARL utilizes leverages Gate Recurrent Unit(GRU) and attention mechanisms to predict cache content based on user historical request information and device details, consequently improving edge server caching performance and users' evaluation of video. Experiment demonstrates that GSRN-MARL surpasses LRU, DRL, and MAAC across two critical metrics: video cache hit rate and users' QoE (Quality of Experience).

关键词： Performance evaluation Training Cloud computing Technological innovation Bit rate Logic gates Mobile handsets Servers Quality of experience Edge computing

来源：评论

学校读者我要写书评

暂无评论

CA-Edit: Causality-Aware Condition Adapter for High-Fidelity Local Facial Attribute Editing 39

CA-Edit: Causality-Aware Condition Adapter for High-Fidelity...

引用

39th Annual AAAI Conference on Artificial Intelligence, AAAI 2025

作者： Xian, Xiaole He, Xilin Niu, Zenghao Zhang, Junliang Xie, Weicheng Song, Siyang Yu, Zitong Shen, Linlin Computer Vision Institute School of Computer Science & Software Engineering Shenzhen University China National Engineering Laboratory for Big Data System Computing Technology Shenzhen University China Guangdong Provincial Key Laboratory of Intelligent Information Processing China University of Exeter United Kingdom Great Bay University China

ISBN: (纸本)157735897X

For efficient and high-fidelity local facial attribute editing, most existing editing methods either require additional finetuning for different editing effects or tend to affect beyond the editing regions. Alternatively, inpainting methods can edit the target image region while preserving external areas. However, current inpainting methods still suffer from the generation misalignment with facial attributes description and the loss of facial skin details. To address these challenges, (i) a novel data utilization strategy is introduced to construct datasets consisting of attribute-text-image triples from a data-driven perspective, (ii) a Causality-Aware Condition Adapter is proposed to enhance the contextual causality modeling of specific details, which encodes the skin details from the original image while preventing conflicts between these cues and textual conditions. In addition, a Skin Transition Frequency Guidance technique is introduced for the local modeling of contextual causality via sampling guidance driven by low-frequency alignment. Extensive quantitative and qualitative experiments demonstrate the effectiveness of our method in boosting both fidelity and editability for localized attribute editing. Our codes will be made publicly available. Copyright © 2025, Association for the Advancement of Artificial Intelligence (***). All rights reserved.

关键词：

来源：评论

学校读者我要写书评

暂无评论

SAV-SE: Scene-aware Audio-Visual Speech Enhancement with Selective State Space Model

引用

IEEE Journal on Selected Topics in Signal Processing 2025年

作者： Qian, Xinyuan Gao, Jiaran Zhang, Yaodan Zhang, Qiquan Liu, Hexin Garcia, Leibny Paola Li, Haizhou University of Science and Technology Beijing School of Computer and Communication Engineering Beijing100083 China University of New South Wales School of Electrical Engineering and Telecommunications Sydney2052 Australia Nanyang Technological University College of Computing and Data Science Singapore Singapore Johns Hopkins University CLSP and HLT-COE United States Chinese University of HongKong Guangdong Provincial Key Laboratory of Big Data Computing Shenzhen518172 China Shenzhen Research Institute of Big data Shenzhen51872 China

Speech enhancement plays an essential role in various applications, and the integration of visual information has been demonstrated to bring substantial advantages. However, the majority of current research concentrates on the examination of facial and lip movements, which can be compromised or entirely inaccessible in scenarios where occlusions occur or when the camera view is distant. Whereas contextual visual cues from the surrounding environment have been overlooked: for example, when we see a dog bark, our brain has the innate ability to discern and filter out the barking noise. To this end, in this paper, we introduce a novel task, i.e. Scene-aware Audio-Visual Speech Enhancement SAV-SE. To our best knowledge, this is the first proposal to use rich contextual information from synchronized video as auxiliary cues to indicate the type of noise, which eventually improves the speech enhancement performance. Specifically, we propose the VC-S $^{2}$ E method, which incorporates the Conformer and Mamba modules for their complementary strengths. Extensive experiments are conducted on public MUSIC, AVSpeech and AudioSet datasets, where the results demonstrate the superiority of VC-S $^{2}$ E over other competitive methods. We will make the source code publicly available. Project demo page: https://***/ © 2007-2012 IEEE.

关键词： Speech enhancement

来源：评论

学校读者我要写书评

暂无评论

Deep Learning-based Image Text Processing Research*

Deep Learning-based Image Text Processing Research*

引用

IEEE International Conference on big data Security on Cloud (bigdataSecurity), High Performance and Smart Computing (HPSC) and Intelligent data and Security (IDS)

作者： Huixuan Xiong Kai Jin Jingnian Liu Jiahong Cai Lijun Xiao School of Computer Science and Engineering Hunan University of Science and Technology Hunan Key Laboratory for Service Computing and Novel Software Technology Xiangtan China the College of Computer Science and Electronics Engineering Hunan University Changsha China Big Data Development and Research Center Guangzhou College of Technology and Business Guangzhou China

Deep learning is a powerful multi-layer architecture that has important applications in image processing and text classification. This paper first introduces the development of deep learning and two important algorithms of deep learning: convolutional neural networks and recurrent neural networks. The paper then introduces three applications of deep learning for image recognition, image detection, and image forensics, as well as three text classification methods based on convolutional neural networks, recurrent neural networks, and other text classification methods. Finally, the development trend of deep learning in the field of text and image processing and the difficulties to be further researched are summarised and prospected.

关键词： Deep learning Image forensics Recurrent neural networks Computational modeling Text categorization Feature extraction data models

来源：评论

学校读者我要写书评

暂无评论

Confidence-aware Contrastive Learning for Selective Classification

arXiv

引用

arXiv 2024年

作者： Wu, Yu-Chang Lyu, Shen-Huan Shang, Haopu Wang, Xiangyu Qian, Chao National Key Laboratory for Novel Software Technology Nanjing University China School of Artificial Intelligence Nanjing University China Key Laboratory of Water Big Data Technology of Ministry of Water Resources Hohai University China College of Computer Science and Software Engineering Hohai University China

Selective classification enables models to make predictions only when they are sufficiently confident, aiming to enhance safety and reliability, which is important in high-stakes scenarios. Previous methods mainly use deep neural networks and focus on modifying the architecture of classification layers to enable the model to estimate the confidence of its prediction. This work provides a generalization bound for selective classification, disclosing that optimizing feature layers helps improve the performance of selective classification. Inspired by this theory, we propose to explicitly improve the selective classification model at the feature level for the first time, leading to a novel Confidence-aware Contrastive Learning method for Selective Classification, CCL-SC, which similarizes the features of homogeneous instances and differentiates the features of heterogeneous instances, with the strength controlled by the model’s confidence. The experimental results on typical datasets, i.e., CIFAR-10, CIFAR-100, CelebA, and ImageNet, show that CCL-SC achieves significantly lower selective risk than state-of-the-art methods, across almost all coverage degrees. Moreover, it can be combined with existing methods to bring further improvement. © 2024, CC BY.

关键词： Classification (of information)

来源：评论

学校读者我要写书评

暂无评论

Unsupervised Community Discovery Algorithm via Reconstructed Graph Neural NEtwork

Unsupervised Community Discovery Algorithm via Reconstructed...

引用

International Conference on Machine Learning and Cybernetics (ICMLC)

作者： Jinghong Wang Hui Wang Jiateng Yang Xizhao Wang College of Computer and Cyber Security Hebei Normal University Shijiazhuang China Hebei Key Laboratory of Network and Information Security Hebei Normal University Shijiazhuang China Hebei Provincial Engineering Research Center for Supply Chain Big Data Analytics & Data Security (Hebei Normal University) Shijiazhuang China College of Artificial Intelligence and Big Data Hebei Polytechnic Institute Shijiazhuang China College of Computer Science and Software Engineering ShenZhen University Shenzhen China

Community discovery can help discover potential community structures in the network, which is a fundamental and important issue in network science. Graph neural network-based algorithms are receiving increasing attention. However, there is relatively little work on community discovery directly using graph neural networks in reconstruction design. This paper presents an Unsupervised Community Discovery Algorithm based on reconstructed graph neural network (UCDA) to resolve the above issues. To be specific, this paper proposes a framework that learns the representation based on network structure and attribute information. By designing input network features, encoders, decoders, this algorithm uses an unsupervised loss function to train neural networks to extract communities in an integrated way. These designs integrate higher-order modularity information with network features and use variational graph auto-encoders to train data end-to-end under the graph neural network framework. Based on a series of experiments with a wide range of datasets and advanced algorithms, UCDA has achieved superior performance and shows strong competitiveness. Finally, the last section reports the results analysis, t-SNE visualization and hyperparameter analysis, which depicts stable performance and powerful network modularity ability.

关键词：

来源：评论

学校读者我要写书评

暂无评论

Deep Reinforcement Learning for Joint Service Placement and Request Scheduling in Mobile Edge Computing Networks

Deep Reinforcement Learning for Joint Service Placement and ...

引用

IEEE Symposium on computers and Communications (ISCC)

作者： Yuxuan Deng Xiuhua Li Chuan Sun Jinlong Hao Xiaofei Wang Victor C. M. Leung School of Big Data & Software Engineering Chongqing University Chongqing China Chongqing Changan Automobile Software Technology Co. Ltd Chongqing China TKLAN College of Intelligence & Computing Tianjin University Tianjin China College of Computer Science & Software Engineering Shenzhen University Shenzhen China Department of Electrical & Computer Engineering The University of British Columbia Vancouver Canada

Mobile edge computing aims to provide cloud-like services on edge servers located near Mobile Devices (MDs) with higher Quality of Service (QoS). However, the mobility of MDs makes it difficult to find a global optimal solution for the coupled service placement and request scheduling problem. To address these issues, we consider a three-tier MEC network with vertical and horizontal cooperation. Then we formulate the joint service placement and request scheduling problem in a mobile scenario with heterogeneous services and resource limits, and convert it into two Markov decision processes to decouple decisions across successive time slots. We propose a Cyclic Deep Q-network-based Service placement and Request scheduling (CDSR) framework to find a long-term optimal solution despite future information unavailability. Specifically, to solve the issue of enormous action space, we decompose the system agent and train them cyclically. Evaluation results demonstrates the effectiveness of our proposed CDSR on user-perceived QoS.

关键词：

来源：评论

学校读者我要写书评

暂无评论

The Role of Depth, Width, and Tree Size in Expressiveness of Deep Forest

arXiv

引用

arXiv 2024年

作者： Lyu, Shen-Huan Wu, Jin-Hui Zheng, Qin-Cheng Ye, Baoliu Key Laboratory of Water Big Data Technology of Ministry of Water Resources Hohai University China College of Computer Science and Software Engineering Hohai University China National Key Laboratory for Novel Software Technology Nanjing University China School of Artificial Intelligence Nanjing University China

Random forests are classical ensemble algorithms that construct multiple randomized decision trees and aggregate their predictions using naive averaging. Zhou and Feng [50] further propose a deep forest algorithm with multi-layer forests, which outperforms random forests in various tasks. The performance of deep forests is related to three hyperparameters in practice: depth, width, and tree size, but little has been known about its theoretical explanation. This work provides the first upper and lower bounds on the approximation complexity of deep forests concerning the three hyperparameters. Our results confirm the distinctive role of depth, which can exponentially enhance the expressiveness of deep forests compared with width and tree size. Experiments confirm the theoretical findings. Copyright © 2024, The Authors. All rights reserved.

关键词： Decision trees

来源：评论

学校读者我要写书评

暂无评论

APADGCN: Adaptive Partial Attention Diffusion Graph Convolutional Network for Traffic Flow Forecasting 4th

APADGCN: Adaptive Partial Attention Diffusion Graph Convolut...

引用

4th International Conference on Spatial data and Intelligence, SpatialDI 2023

作者： Zhang, Bowen Li, Bohan Wei, Jinzhan Wen, Hao College of Computer Science and Technology Nanjing University of Aeronautics and Astronautics Nanjing211106 China Key Laboratory of Safety-Critical Software Ministry of Industry and Information Technology Beijing China National Engineering Laboratory for Integrated Aero-Space-Ground Ocean Big Data Application Technology Xi’an China School of Electronic Information and Automation Guilin University of Aerospace Technology Guilin541010 China College of Aerospace Engineering Nanjing University of Aeronautics and Astronautics Nanjing211106 China

ISBN: (纸本)9783031329098

Traffic flow forecasting is a core task of urban governance and plays a vital role in the development of ITS. Because of the complexity and uncertainty of traffic patterns, it is of great challenge to capture spatial-temporal correlations. Recent researches mainly focus on the pre-defined adjacency matrix based on prior knowledge as the basis of spatial-temporal correlation modeling, but the fixed graph structure cannot adequately describe the dependency between traffic sensors. To tackle this issue, a novel deep learning model framework is proposed in this paper: Adaptive Partial Attention Diffusion Graph Convolutional Network(APADGCN), which consists of three main parts: 1) the Multi-Component module that divides the historical traffic flow into recent, daily-periodic, and weekly-periodic, to capture the traffic patterns of different periodic;2) the spatial correlation modeling which can dynamically capture node relationships and model spatial dependency, and enhance the aggregation ability of low-order information;3) the temporal correlation modeling which models long-term time dependencies using convolution and gating. The final result is obtained by the weighted fusion of the results of the multi-components. We compared our APADGCN with various baseline models in the four real datasets from the Caltrans Performance Measurement System (PeMS). The experimental results show that the prediction accuracy of APADGCN is better than that of the baseline models. © 2023, The Author(s), under exclusive license to Springer Nature Switzerland AG.

关键词： Convolution

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：