检索结果-内蒙古大学图书馆

Stochastic normalized gradient descent with momentum for large-batch training

science China(Information sciences) 2024年第11期67卷 77-91页

作者： Shen-Yi ZHAO Chang-Wei SHI Yin-Peng XIE Wu-Jun LI National Key Laboratory for Novel Software Technology Department of Computer Science and TechnologyNanjing University

Stochastic gradient descent(SGD) and its variants have been the dominating optimization methods in machine learning. Compared with SGD with small-batch training, SGD with large-batch training can better utilize the computational power of current multi-core systems such as graphics processing units(GPUs)and can reduce the number of communication rounds in distributed training settings. Thus, SGD with large-batch training has attracted considerable attention. However, existing empirical results showed that large-batch training typically leads to a drop in generalization accuracy. Hence, how to guarantee the generalization ability in large-batch training becomes a challenging task. In this paper, we propose a simple yet effective method, called stochastic normalized gradient descent with momentum(SNGM), for large-batch training. We prove that with the same number of gradient computations, SNGM can adopt a larger batch size than momentum SGD(MSGD), which is one of the most widely used variants of SGD, to converge to an?-stationary point. Empirical results on deep learning verify that when adopting the same large batch size,SNGM can achieve better test accuracy than MSGD and other state-of-the-art large-batch training methods.

关键词： non-convex problems large-batch training stochastic normalized gradient descent momentum

来源：评论

学校读者我要写书评

暂无评论

Re-quantization based binary graph neural networks

引用

science China(Information sciences) 2024年第7期67卷 160-171页

作者： Kai-Lang YAO Wu-Jun LI National Key Laboratory for Novel Software Technology Department of Computer Science and TechnologyNanjing University

Binary neural networks have become a promising research topic due to their advantages of fast inference speed and low energy consumption. However, most existing studies focus on binary convolutional neural networks, while less attention has been paid to binary graph neural networks. A common drawback of existing studies on binary graph neural networks is that they still include lots of inefficient full-precision operations in multiplying three matrices and are therefore not efficient enough. In this paper, we propose a novel method, called re-quantization-based binary graph neural networks(RQBGN), for binarizing graph neural networks. Specifically, re-quantization, a necessary procedure contributing to the further reduction of superfluous inefficient full-precision operations, quantizes the results of multiplication between any two matrices during the process of multiplying three matrices. To address the challenges introduced by requantization, in RQBGN we first study the impact of different computation orders to find an effective one and then introduce a mixture of experts to increase the model capacity. Experiments on five benchmark datasets show that performing re-quantization in different computation orders significantly impacts the performance of binary graph neural network models, and RQBGN can outperform other baselines to achieve state-of-the-art performance.

关键词： graph neural networks binary neural networks mixture of experts computation-efficient algorithms

来源：评论

学校读者我要写书评

暂无评论

When debugging encounters artificial intelligence: state of the art and open challenges

引用

science China(Information sciences) 2024年第4期67卷 42-84页

作者： Yi SONG Xiaoyuan XIE Baowen XU School of Computer Science Wuhan University State Key Laboratory of Novel Software Technology Nanjing University

Both software debugging and artificial intelligence techniques are hot topics in the current field of software engineering. Debugging techniques, which comprise fault localization and program repair, are an important part of the software development lifecycle for ensuring the quality of software systems. As the scale and complexity of software systems grow, developers intend to improve the effectiveness and efficiency of software debugging via artificial intelligence(artificial intelligence for software debugging, AI4SD). On the other hand, many artificial intelligence models are being integrated into safety-critical areas such as autonomous driving, image recognition, and audio processing, where software debugging is highly necessary and urgent(software debugging for artificial intelligence, SD4AI). An AI-enhanced debugging technique could assist in debugging AI systems more effectively, and a more robust and reliable AI approach could further guarantee and support debugging techniques. Therefore, it is important to take AI4SD and SD4AI into consideration comprehensively. In this paper, we want to show readers the path, the trend, and the potential that these two directions interact with each other. We select and review a total of 165 papers in AI4SD and SD4AI for answering three research questions, and further analyze opportunities and challenges as well as suggest future directions of this cross-cutting area.

关键词： software debugging fault localization program repair artificial intelligence machine learning

来源：评论

学校读者我要写书评

暂无评论

Understanding and Detecting Inefficient Image Displaying Issues in Android Apps

引用

Journal of computer science & technology 2024年第2期39卷 434-459页

作者：李文杰马骏蒋炎岩许畅马晓星 State Key Laboratory of Novel Software Technology Nanjing UniversityNanjing 210023China Department of Computer Science and Technology Nanjing UniversityNanjing 210023China

Mobile applications(apps for short)often need to display ***,inefficient image displaying(IID)issues are pervasive in mobile apps,and can severely impact app performance and user *** paper first establishes a descriptive framework for the image displaying procedures of IID *** on the descriptive framework,we conduct an empirical study of 216 real-world IID issues collected from 243 popular open-source Android apps to validate the presence and severity of IID issues,and then shed light on these issues’characteristics to support research on effective issue *** the findings of this study,we propose a static IID issue detection tool TAPIR and evaluate it with 243 real-world Android ***,49 and 64 previously-unknown IID issues in two different versions of 16 apps reported by TAPIR are manually confirmed as true positives,respectively,and 16 previously-unknown IID issues reported by TAPIR have been confirmed by developers and 13 have been ***,we further evaluate the performance impact of these detected IID issues and the performance improvement if they are *** results demonstrate that the IID issues detected by TAPIR indeed cause significant performance degradation,which further show the effectiveness and efficiency of TAPIR.

关键词： Android application(app) inefficient image displaying(IID) performance empirical study static analysis

来源：评论

学校读者我要写书评

暂无评论

An Intelligent Privacy Protection Scheme for Efficient Edge Computation Offloading in IoV

引用

Chinese Journal of Electronics 2024年第4期33卷 910-919页

作者： Liang YAO Xiaolong XU Wanchun DOU Muhammad Bilal School of Software Nanjing University of Information Science and Technology State Key Laboratory for Novel Software Technology Nanjing University Department of Computer and Electronics Systems Engineering Hankuk University of Foreign Studies

As a pivotal enabler of intelligent transportation system(ITS), Internet of vehicles(Io V) has aroused extensive attention from academia and industry. The exponential growth of computation-intensive, latency-sensitive,and privacy-aware vehicular applications in Io V result in the transformation from cloud computing to edge computing,which enables tasks to be offloaded to edge nodes(ENs) closer to vehicles for efficient execution. In ITS environment,however, due to dynamic and stochastic computation offloading requests, it is challenging to efficiently orchestrate offloading decisions for application requirements. How to accomplish complex computation offloading of vehicles while ensuring data privacy remains challenging. In this paper, we propose an intelligent computation offloading with privacy protection scheme, named COPP. In particular, an Advanced Encryption Standard-based encryption method is utilized to implement privacy protection. Furthermore, an online offloading scheme is proposed to find optimal offloading policies. Finally, experimental results demonstrate that COPP significantly outperforms benchmark schemes in the performance of both delay and energy consumption.

关键词： Industries Privacy Energy consumption Transportation Computational efficiency Encryption Protection

来源：评论

学校读者我要写书评

暂无评论

A program logic for obstruction-freedom

引用

Frontiers of computer science 2024年第6期18卷 85-100页

作者： Zhao-Hui LI Xin-Yu FENG School of Computer Science and Technology University of Science and Technology of ChinaHefei 230026China Department of Computer Science and Technology Nanjing UniversityNanjing 210023China State Key Laboratory for Novel Software Technology Nanjing UniversityNanjing 210023China

Though obstruction-free progress property is weaker than other non-blocking properties including lock-freedom and wait-freedom,it has advantages that have led to the use of obstruction-free implementations for software transactional memory(STM)and in anonymous and fault-tolerant distributed ***,existing work can only verify obstruction-freedom of specific data structures(e.g.,STM and list-based algorithms).In this paper,to fill this gap,we propose a program logic that can formally verify obstruction-freedom of practical implementations,as well as verify linearizability,a safety property,at the same *** also propose informal principles to extend a logic for verifying linearizability to verifying *** this approach,the existing proof for linearizability can be reused directly to construct the proof for both linearizability and ***,we have successfully applied our logic to verifying a practical obstruction-free double-ended queue implementation in the first classic paper that has proposed the definition of obstruction-freedom.

关键词： verification program logic progress properties obstruction-freedom concurrent objects

来源：评论

学校读者我要写书评

暂无评论

An Empirical Study on Automated Test Generation Tools for Java:Effectiveness and Challenges

引用

Journal of computer science & technology 2024年第3期39卷 715-736页

作者：刘相君余萍马晓星 State Key Laboratory for Novel Software Technology Nanjing UniversityNanjing 210023China Department of Computer Science and Technology Nanjing UniversityNanjing 210023China CCF ACM IEEE

Automated test generation tools enable test automation and further alleviate the low efficiency caused by writing hand-crafted test ***,existing automated tools are not mature enough to be widely used by software testing *** paper conducts an empirical study on the state-of-the-art automated tools for Java,i.e.,EvoSuite,Randoop,JDoop,JTeXpert,T3,and *** design a test workflow to facilitate the process,which can automatically run tools for test generation,collect data,and evaluate various ***,we conduct empirical analysis on these six tools and their related techniques from different aspects,i.e.,code coverage,mutation score,test suite size,readability,and real fault detection *** discuss about the benefits and drawbacks of hybrid techniques based on experimental ***,we introduce our experience in setting up and executing these tools,and summarize their usability and ***,we give some insights into automated tools in terms of test suite readability improvement,meaningful assertion generation,test suite reduction for random testing tools,and symbolic execution integration.

关键词： automated test generation search-based software testing random testing symbolic execution

来源：评论

学校读者我要写书评

暂无评论

Machine learning automation

引用

National science Review 2024年第8期11卷 7-8页

作者： Zongben Xu Zhi-Hua Zhou Wenwu Zhu School of Mathematics and Statistics Xi'an Jiaotong University National Key Laboratory for Novel Software Technology Nanjing University Department of Computer Science and Technology Tsinghua University

With the exponential growth of big data and advancements in large-scale foundation model techniques, the field of machine learning has embarked on an unprecedented golden era. This period is characterized by significant innovations across various aspects of machine learning, including data exploitation, network architecture development, loss function settings and algorithmic innovation.

关键词： automation learning machine

来源：评论

学校读者我要写书评

暂无评论

Clustered Reinforcement Learning

引用

Frontiers of computer science 2025年第4期19卷 43-57页

作者： Xiao MA Shen-Yi ZHAO Zhao-Heng YIN Wu-Jun LI National Key Laboratory for Novel Software Technology Department of Computer Science and TechnologyNanjing UniversityNanjing 210023China Department of Electrical Engineering and Computer Sciences University of CaliforniaBerkeleyCA 94720-1770USA

Exploration strategy design is a challenging problem in reinforcement learning(RL),especially when the environment contains a large state space or sparse *** exploration,the agent tries to discover unexplored(novel)areas or high reward(quality)*** existing methods perform exploration by only utilizing the novelty of *** novelty and quality in the neighboring area of the current state have not been well utilized to simultaneously guide the agent’s *** address this problem,this paper proposes a novel RL framework,called clustered reinforcement learning(CRL),for efficient exploration in *** adopts clustering to divide the collected states into several clusters,based on which a bonus reward reflecting both novelty and quality in the neighboring area(cluster)of the current state is given to the *** leverages these bonus rewards to guide the agent to perform efficient ***,CRL can be combined with existing exploration strategies to improve their performance,as the bonus rewards employed by these existing exploration strategies solely capture the novelty of *** on four continuous control tasks and six hard-exploration Atari-2600 games show that our method can outperform other state-of-the-art methods to achieve the best performance.

关键词： deep reinforcement learning exploration count-based method clustering K-means

来源：评论

学校读者我要写书评

暂无评论

A survey on model-based reinforcement learning

引用

science China(Information sciences) 2024年第2期67卷 59-84页

作者： Fan-Ming LUO Tian XU Hang LAI Xiong-Hui CHEN Weinan ZHANG Yang YU National Key Laboratory for Novel Software Technology Nanjing University Polixir. ai Department of Computer Science and Engineering Shanghai Jiao Tong University

Reinforcement learning(RL) interacts with the environment to solve sequential decision-making problems via a trial-and-error approach. Errors are always undesirable in real-world applications, even though RL excels at playing complex video games that permit several trial-and-error attempts. To improve sample efficiency and thus reduce errors, model-based reinforcement learning(MBRL) is believed to be a promising direction, as it constructs environment models in which trial-and-errors can occur without incurring actual costs. In this survey, we investigate MBRL with a particular focus on the recent advancements in deep RL. There is a generalization error between the learned model of a non-tabular environment and the actual environment. Consequently, it is crucial to analyze the disparity between policy training in the environment model and that in the actual environment, guiding algorithm design for improved model learning, model utilization, and policy training. In addition, we discuss the recent developments of model-based techniques in other forms of RL, such as offline RL, goal-conditioned RL, multi-agent RL, and meta-RL. Furthermore,we discuss the applicability and benefits of MBRL for real-world tasks. Finally, this survey concludes with a discussion of the promising future development prospects for MBRL. We believe that MBRL has great unrealized potential and benefits in real-world applications, and we hope this survey will encourage additional research on MBRL.

关键词： reinforcement learning model-based reinforcement learning planning model learning model learning with reduced error model usage

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：