检索结果-内蒙古大学图书馆

ROI-HIT: Region of Interest-Driven High-Dimensional Microarchitecture Design Space Exploration

IEEE TRANSACTIONS ON COMPUTER-AIDED DESIGN OF INTEGRATED CIRCUITS AND SYSTEMS 2024年第11期43卷 4178-4189页

作者： Zhao, Xuyang Gao, Tianning Zhao, Aidong Bi, Zhaori Yan, Changhao Yang, Fan Wang, Sheng-Guo Zhou, Dian Zeng, Xuan Fudan Univ Sch Microelect State Key Lab Integrated Chips & Syst Shanghai 200433 Peoples R China Univ Texas Dallas Dept Elect Engn Dallas TX 75080 USA Univ North Carolina Charlotte Coll Engn Charlotte NC 28223 USA

Exploring the design space of RISC-V processors faces significant challenges due to the vastness of the high-dimensional design space and the associated expensive simulation costs. This work proposes a region of interest (ROI)-driven method, which focuses on the promising ROIs to reduce the over-exploration on the huge design space and improve the optimization efficiency. A tree structure based on self-organizing map (SOM) networks is proposed to partition the design space into ROIs. To reduce the high dimensionality of design space, a variable selection technique based on a sensitivity matrix is developed to prune unimportant design parameters and efficiently hit the optimum inside the ROIs. Moreover, an asynchronous parallel strategy is employed to further save the time taken by simulations. Experimental results demonstrate the superiority of our proposed method, achieving improvements of up to 43.82% in performance, 33.20% in power consumption, and 11.41% in area compared to state-of-the-art methods.

关键词： asynchronous parallel optimization high-dimensional design space exploration (DSE) region of interest (ROI) RISC-V microarchitecture variable selection (VS)

来源：评论

学校读者我要写书评

暂无评论

On the parallelization Upper Bound for asynchronous Stochastic Gradients Descent in Non-convex optimization

引用

JOURNAL OF optimization THEORY AND APPLICATIONS 2023年第3期196卷 900-935页

作者： Wang, Lifu Shen, Bo Beijing Jiaotong Univ Sch Elect & Informat Engn Beijing Peoples R China Beijing Jiaotong Univ Key Lab Commun & Informat Syst Beijing Municipal Commiss Educ Beijing Peoples R China

In deep learning, asynchronous parallel stochastic gradient descent (APSGD) is a broadly used algorithm to speed up the training process. In asynchronous system, the time delay of stale gradients in asynchronous algorithms is generally proportional to the total number of workers. When the number of workers is too large, the time delay will bring additional deviation from the accurate gradient due to delayed gradients and may cause a negative influence on the convergence speed of the algorithm. One may ask: How many workers can be used at most to achieve both the convergence and the speedup? In this paper, we consider the asynchronous training problem with the non-convex case. We theoretically study this problem to find an approximating second-order stationary point using asynchronous algorithms in non-convex optimization and investigate the behaviors of APSGD near-saddle points. This work gives the first theoretical guarantee to find an approximating second-order stationary point in asynchronous algorithms and a provable upper bound for the time delay. The techniques we provide can be generalized to analyze other types of asynchronous algorithms to understand the behaviors of asynchronous algorithms in distributed asynchronous parallel training.

关键词： Non-convex optimization asynchronous parallel optimization Saddle points escaping Deep learning

来源：评论

学校读者我要写书评

暂无评论

asynchronous Vertical Federated Learning for Kernelized AUC Maximization 24

Asynchronous Vertical Federated Learning for Kernelized AUC ...

引用

30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining

作者： Zhang, Ke Wang, Ganyu Li, Han Wang, Yulong Chen, Hong Gu, Bin Huazhong Agr Univ Coll Informat Wuhan Hubei Peoples R China Western Univ Dept Comp Sci London ON Canada Engn Res Ctr Intelligent Technol Agr Wuhan Hubei Peoples R China Jilin Univ Sch Artificial Intelligence Changchun Jilin Peoples R China Mohamed Bin Zayed Univ Artificial Intelligence Masdar U Arab Emirates

ISBN: (纸本)9798400704901

Vertical Federated Learning (VFL) has garnered significant attention due to its applicability in multi-party collaborative learning and the increasing demand for privacy-preserving measures. Most existing VFL algorithms primarily focus on accuracy as the training model metric. However, the data we access is often imbalanced in the real world, making it difficult for models based on accuracy to correctly classify minority samples. The Area Under the Curve (AUC) serves as an effective metric to evaluate the performance of a model on imbalanced data. Therefore, optimizing AUC can enhance the model's ability to handle imbalanced data. Besides, computational resources within VFL systems are also imbalanced, which makes synchronous VFL algorithms are difficult to apply in the real world. To address the double imbalance issue, we propose asynchronous Vertical Federated Kernelized AUC Maximization (AVFKAM). Specifically, AVFKAM asynchronously updates a kernel model based on triply stochastic gradients with respect to (w.r.t) the pairwise loss and random feature approximation. To facilitate theoretical analysis, we transfer the asynchrony of model coefficients to the functional gradient through a dual relationship between coefficients and objective function. Furthermore, we demonstrate that AVFKAM converges to the optimal solution at a rate of O( 1/t), where t represents the global iteration number, and discuss the security of the model. Finally, experimental results on various benchmark datasets demonstrate that AVFKAM maintains high AUC performance and efficiency.

关键词： AUC maximization asynchronous parallel optimization vertical federated learning kernel methods

来源：评论

学校读者我要写书评

暂无评论

On the rates of convergence of parallelized averaged stochastic gradient algorithms

引用

STATISTICS 2020年第3期54卷 618-635页

作者： Godichon-Baggioni, Antoine Saadane, Sofiane INSA Rouen Lab Math F-76800 St Etienne Du Rouvray France INSA Toulouse Inst Math Toulouse Toulouse France

The growing interest for high-dimensional and functional data analysis led in the last decade to important research developing a consequent amount of techniques. parallelized algorithms, which consist of distributing and treat the data into different machines, for example, are a good answer to deal with large samples taking values in high-dimensional spaces. We introduce here a parallelized averaged stochastic gradient algorithm, which enables to treat efficiently and recursively the data, and so, without taking care if the distribution of the data into the machines is uniform. The rate of convergence in quadratic mean, as well as the asymptotic normality of the parallelized estimates are given, for strongly and locally strongly convex objectives.

关键词： Stochastic gradient descent averaging distributed estimation central limit theorem asynchronous parallel optimization

来源：评论

学校读者我要写书评

暂无评论

Online Feature Selection: A Limited-Memory Substitution Algorithm and Its asynchronous parallel Variation 16

Online Feature Selection: A Limited-Memory Substitution Algo...

引用

22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD)

作者： Yang, Haichuan Fujimaki, Ryohei Kusumura, Yukitaka Liu, Ji Univ Rochester Dept Comp Sci Rochester NY 14627 USA NEC Corp Ltd Cupertino CA 95014 USA

ISBN: (纸本)9781450342322

This paper considers the feature selection scenario where only a few features are accessible at any time point. For example, features are generated sequentially and visible one by one. Therefore, one has to make an online decision to identify key features after all features are only scanned once or twice. The optimization based approach is a powerful tool for the online feature selection. However, most existing optimization based algorithms explicitly or implicitly adopt L-1 norm regularization to identify important features, and suffer two main disadvantages: 1) the penalty term for L-1 norm term is hard to choose;and 2) the memory usage is hard to control or predict. To overcome these two drawbacks, this paper proposes a limited-memory and model parameter free online feature selection algorithm, namely online substitution (OS) algorithm. To improve the selection efficiency, an asynchronous parallel extension for OS (Asy-OS) is proposed. Convergence guarantees are provided for both algorithms. Empirical study suggests that the performance of OS and Asy-OS is comparable to the benchmark algorithm Grafting, but requires much less memory cost and can be easily extended to the parallel implementation.

关键词： Feature Selection Online Learning asynchronous parallel optimization

来源：评论

学校读者我要写书评

暂无评论

An asynchronous parallel Stochastic Coordinate Descent Algorithm

引用

JOURNAL OF MACHINE LEARNING RESEARCH 2015年第1期16卷 285-322页

作者： Liu, Ji Wright, Stephen J. Re, Christopher Bittorf, Victor Sridhar, Srikrishna Univ Wisconsin Madison Dept Comp Sci Madison WI 53706 USA Stanford Univ Dept Comp Sci Stanford CA 94305 USA

We describe an asynchronous parallel stochastic coordinate descent algorithm for minimizing smooth unconstrained or separably constrained functions. The method achieves a linear convergence rate on functions that satisfy an essential strong convexity property and a sublinear rate (1/K) on general convex functions. Near-linear speedup on a multicore system can be expected if the number of processors is O (n(1/2)) in unconstrained optimization and O (n(1/4)) in the separable-constrained case, where n is the number of variables. We describe results from implementation on 40-core processors.

关键词： asynchronous parallel optimization stochastic coordinate descent

来源：评论

学校读者我要写书评

暂无评论

An asynchronous parallel stochastic coordinate descent algorithm

引用

The Journal of Machine Learning Research 2015年第1期16卷

作者： Ji Liu Stephen J. Wright Christopher Ré Victor Bittorf Srikrishna Sridhar Department of Computer Sciences University of Wisconsin-Madison Madison WI Department of Computer Science Stanford University Stanford CA

We describe an asynchronous parallel stochastic coordinate descent algorithm for minimizing smooth unconstrained or separably constrained functions. The method achieves a linear convergence rate on functions that satisfy an essential strong convexity property and a sublinear rate (1/K) on general convex functions. Near-linear speedup on a multicore system can be expected if the number of processors is O(n1/2) in unconstrained optimization and O(n1/4) in the separable-constrained case, where n is the number of variables. We describe results from implementation on 40-core processors.

关键词： asynchronous parallel optimization stochastic coordinate descent

来源：评论

学校读者我要写书评

暂无评论

Sprouting search - an algorithmic framework for asynchronous parallel unconstrained optimization

引用

optimization METHODS & SOFTWARE 2007年第5期22卷 755-776页

作者： Burmen, Arpad Tuma, Tadej Univ Ljubljana Fac Elect Engn SI-1000 Ljubljana Slovenia

Direct search optimization algorithms are becoming an important alternative to well- established gradient based methods. Due to the fact that a single cost function evaluation may take a substantial amount of time, optimization can be a lengthy process. In order to shorten the run time one often resorts to parallel algorithms. asynchronous algorithms are particularly efficient since they have no synchronization points. This paper is an attempt to establish a convergence theory for a class of such parallel direct search algorithms. The notion of a search direction generator (SDG) is introduced. An algorithmic framework for parallel distributed optimization methods based on SDGs is presented along with the corresponding convergence theory. The theory almost completely decouples the stepsize control from the sufficient descent requirement, which is necessary for the finite termination of the algorithm's inner loop. The proposed framework has several attributes considered very favourable in loosely coupled parallel systems (e. g. clusters of workstations), such as fault tolerance and scalability. The framework is illustrated by optimizing a set of test problems on a cluster of workstations. In all tested cases, a speedup was obtained that increased with the increasing number of workstations. Fault tolerance and scalability of the framework were also demonstrated by removing and adding workstations to the cluster while an optimization run was in progress.

关键词： asynchronous parallel optimization direct search convergence analysis distributed computing cluster computing

来源：评论

学校读者我要写书评

暂无评论

asynchronous parallel generating set search for linearly constrained optimization

引用

SIAM JOURNAL ON SCIENTIFIC COMPUTING 2008年第4期30卷 1892-1924页

作者： Griffin, Joshua D. Kolda, Tamara G. Lewis, Robert Michael Sandia Natl Labs Computat Sci & Mat Res Dept Livermore CA 94551 USA Coll William & Mary Dept Mat Williamsburg VA 23187 USA

We describe an asynchronous parallel derivative-free algorithm for linearly constrained optimization. Generating set search (GSS) is the basis of our method. At each iteration, a GSS algorithm computes a set of search directions and corresponding trial points and then evaluates the objective function value at each trial point. asynchronous versions of the algorithm have been developed in the unconstrained and bound-constrained cases which allow the iterations to continue (and new trial points to be generated and evaluated) as soon as any other trial point completes. This enables better utilization of parallel resources and a reduction in overall run time, especially for problems where the objective function takes minutes or hours to compute. For linearly constrained GSS, the convergence theory requires that the set of search directions conforms to the nearby boundary. This creates an immediate obstacle for asynchronous methods where the definition of nearby is not well defined. In this paper, we develop an asynchronous linearly constrained GSS method that overcomes this difficulty and maintains the original convergence theory. We describe our implementation in detail, including how to avoid function evaluations by caching function values and using approximate lookups. We test our implementation on every CUTEr test problem with general linear constraints and up to 1000 variables. Without tuning to individual problems, our implementation was able to solve 95% of the test problems with 10 or fewer variables, 73% of the problems with 11-100 variables, and nearly half of the problems with 100-1000 variables. To the best of our knowledge, these are the best results that have ever been achieved with a derivative-free method for linearly constrained optimization. Our asynchronous parallel implementation is freely available as part of the APPSPACK software.

关键词： nonlinear programming constrained optimization linear constraints direct search derivative-free optimization generalized pattern search generating set search asynchronous parallel optimization asynchronous parallel pattern search

来源：评论

学校读者我要写书评

暂无评论

Revisiting asynchronous parallel pattern search for nonlinear optimization

引用

SIAM JOURNAL ON optimization 2005年第2期16卷 563-586页

作者： Kolda, TG Sandia Natl Labs Computat Sci & Math Res Dept Livermore CA 94551 USA

We present a new asynchronous parallel pattern search (APPS) method which is different from that developed previously by Hough, Kolda, and Torczon. APPS efficiently uses parallel and distributed computing platforms to solve science and engineering design optimization problems where derivatives are unavailable and cannot be approximated. The original APPS was designed to be fault-tolerant as well as asynchronous and was based on a peer-to-peer design. Each process was in charge of a single, fixed search direction. Our new version is based instead on a manager-worker paradigm. Though less fault-tolerant, the resulting algorithm is more flexible in its use of distributed computing resources. We further describe how to incorporate a zero-order sufficient decrease condition and handle bound constraints. Convergence theory for all situations (unconstrained and bound constrained as well as simple and sufficient decrease) is developed. We close with a discussion of how the new APPS will better facilitate the future incorporation of linear and nonlinear constraints.

关键词： asynchronous parallel optimization pattern search direct search distributed computing generating set search

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：