检索结果-内蒙古大学图书馆

IEEE International Conference on Cluster Computing

作者： Alexander V. Goponenko Ramin Izadpanah Jim M. Brandt Damian Dechev 211 Harris Center (Building 116) Orlando FL Sandia National Laboratories Albuquerque MS

ISBN: (数字)9781728166773

ISBN: (纸本)9781728166780

The performance of HPC clusters depends on efficient scheduling of jobs. However, modern schedulers generally lack real-time information about resource utilization and require users to provide information, which is seldom accurate, on job requirements. The problem is exacerbated as HPC systems become increasingly more complicated and heterogeneous, which gives rise to new resource constraints (GPU, parallel file system, network bandwidth, burst buffers, etc.) In this work, we integrated data from LDMS, the Lightweight distributed Metric Service, with Slurm, a popular job scheduler. To demonstrate the capabilities of such integration, we enabled scheduling based on the Lustre file system throughput. We demonstrated benefits of measurement of real-time utilization, prediction of applications requirements from historical data, and finer control of resources, in a preliminary evaluation of scheduling on a cluster of virtual machines. We also identified the possibility of further improving the scheduling efficiency through workload-adaptive scheduling, by adjusting the scheduling based on characteristics of the pending job. We validated the feasibility of this strategy by simulating job executions in our custom-made HPC cluster simulator.

关键词： Measurement File systems Graphics processing units distributed databases Throughput real-time systems Resource management

来源：评论

学校读者我要写书评

暂无评论

Efficient Data Placement and Replication for QoS-Aware Approximate Query Evaluation of Big Data Analytics

引用

IEEE TRANSACTIONS ON parallel AND distributed systems 2019年第12期30卷 2677-2691页

作者： Xia, Qiufen Xu, Zichuan Liang, Weifa Yu, Shui Guo, Song Zomaya, Albert Y. Dalian Univ Technol Key Lab Ubiquitous Network & Serv Software Liaoni Int Sch Informat Sci & Engn Dalian 116024 Liaoning Peoples R China Dalian Univ Technol Sch Software Dalian 116024 Liaoning Peoples R China Australian Natl Univ Res Sch Comp Sci Canberra ACT 2601 Australia Univ Technol Sydney Sch Software Ultimo NSW 2007 Australia Hong Kong Polytech Univ Dept Comp Hung Hom Hong Kong Peoples R China Univ Sydney Sch Comp Sci Camperdown NSW 2006 Australia

Enterprise users at different geographic locations generate large-volume data that is stored at different geographic datacenters. These users may also perform big data analytics on the stored data to identify valuable information in order to make strategic decisions. However, it is well known that performing big data analytics on data in geographical-located datacenters usually is time-consuming and costly. In some delay-sensitive applications, the query result may become useless if answering a query takes too long time. Instead, sometimes users may only be interested in timely approximate rather than exact query results. When such approximate query evaluation is the case, applications must sacrifice timeliness to get more accurate evaluation results or tolerate evaluation result with a guaranteed error bound obtained from analyzing the samples of the data to meet their stringent timeline. In this paper, we study quality-of-service (QoS)-aware data replication and placement for approximate query evaluation of big data analytics in a distributed cloud, where the original (source) data of a query is distributed at different geo-distributed datacenters. We focus on the problems of placing data samples of the source data at some strategic datacenters to meet stringent query delay requirements of users, by exploring a non-trivial trade-off between the cost of query evaluation and the error bound of the evaluation result. We first propose an approximation algorithm with a provable approximation ratio for a single approximate query. We then develop an efficient heuristic algorithm for evaluating a set of approximate queries with the aim to minimize the evaluation cost while meeting the delay requirements of these queries. We finally demonstrate the effectiveness and efficiency of the proposed algorithms through both experimental simulations and implementations in a real test-bed, real datasets are employed. Experimental results show that the proposed algorithms are promisi

关键词： Big Data Query processing Delays Approximation algorithms Quality of service distributed databases Software Data replication and placement big data analytics approximate query evaluation approximation algorithms algorithm analysis

来源：评论

学校读者我要写书评

暂无评论

Coded Matrix Multiplication on a Group-Based Model

Coded Matrix Multiplication on a Group-Based Model

引用

IEEE International Symposium on Information Theory (ISIT)

作者： Kim, Muah Sohn, Jy-yong Moon, Jackyun Korea Adv Inst Sci & Technol Sch Elect Engn Daejeon South Korea

ISBN: (纸本)9781538692912

Coded distributed computing has been considered as a promising technique which makes large-scale systems robust to the "straggler" workers. Yet, practical system models for distributed computing have not been available that reflect the clustered or grouped structure of real-world computing servers. Also, the large variations in the computing power and bandwidth capabilities across different servers have not been properly modeled. We suggest a group-based model to reflect practical conditions and develop an appropriate coding scheme for this model. The suggested code, called group code, employs parallel encoding for each group. We show that the suggested coding scheme can asymptotically achieve optimal computing time in the regime of infinite n, the number of workers. While theoretical analysis is conducted in the asymptotic regime, numerical results also show that the suggested scheme achieves near-optimal computing time for any finite but reasonably large n. Moreover, we demonstrate that decoding complexity of the suggested scheme is significantly reduced by the virtue of parallel decoding.

关键词： Computational modeling Decoding Task analysis Encoding Complexity theory distributed computing Resource management

来源：评论

学校读者我要写书评

暂无评论

On the Feasibility of Hybrid Electrical/Optical Switch Architecture for Large-Scale Training of distributed Deep Learning

On the Feasibility of Hybrid Electrical/Optical Switch Archi...

引用

IEEE/ACM workshop on Photonics-Optics Technology Oriented Networking, Information and Computing systems (PHOTONICS)

作者： Thao Nguyen Truong Takano, Ryousei Natl Inst Adv Ind Sci & Technol AIST Tokyo Tech Real World Big Data Computat Open Meguro Ku 2-12-1 W8-92 Ookayama Tokyo 1528550 Japan

ISBN: (纸本)9781728159812

Data parallelism is the dominant method used to train deep learning (DL) model on High-Performance Computing systems such as large-scale GPU clusters. When training a DL model on a large number of nodes, inter-node communication becomes bottle-neck due to its relatively higher latency and lower link bandwidth (than intra-node communication). To cope with this problem, some techniques have been proposed to (a) optimize the collective communication algorithms that take into account the network topology, (b) reduce the message size, and (c) overlap the communication and computation. All of these approaches target to deal with the large message size issue while diminishing the effect of the limitation of the inter-node network. In this study, we investigate the benefit of increasing inter-node link bandwidth by using the hybrid switching systems, i.e., Electrical Packet Switching and Optical Circuit Switching. We found that the typical data-transfer of synchronous data-parallelism training are long-live and rarely changed that can be speed-up with optical switching. Simulation results on Simgrid simulator show that our approach speed-up the training time of deep learning application around 10%.

关键词： distributed Deep Learning High Performance Computing (HPC) Optical Circuit Switching Hybrid Switching

来源：评论

学校读者我要写书评

暂无评论

Towards task-based parallelization for entity resolution

引用

SICS SOFTWARE-INTENSIVE CYBER-PHYSICAL systems 2020年第1-2期35卷 31-38页

作者： Gazzarri, Leonardo Herschel, Melanie Univ Stuttgart Inst Parallel & Distributed Syst Stuttgart Germany

Entity resolution (ER) refers to the problem of finding which virtual representations in one or more data sources refer to the same real-world entity. A central question in ER is how to find matching entity representations (so called duplicates) efficiently and in a scalable way. One general technique to address these issues is to leverage parallelization. In particular, almost all work on parallel ER focus on data parallelism. This paper focuses on task parallelism for ER. This type of parallelism allows to support incremental ER that offers incremental computation of the solution by streaming results of intermediate stages of ER as soon as they are computed. This possibly allows to obtain results in a more timely fashion and can also serve in a service-oriented setting with limited time or monetary budget. In summary, this paper presents a framework for task-parallelization of ER, supporting in particular ER of large amounts of semi-structured and heterogeneous data. We also discuss a possible implementation of our framework.

关键词： Entity resolution Task-parallelism Blocking parallelization

来源：评论

学校读者我要写书评

暂无评论

distributed computing for construction project design by division into project design units 17

Distributed computing for construction project design by div...

引用

17th International workshop on Intelligent Computing in Engineering, EG-ICE 2010

作者： Suprun, Anatoliy Kislitsyn, Dmitriy Nizhny Novgorod State University of Architecture and Civil Engineering Russia

ISBN: (纸本)9781907284601

A new technique is proposed for constructing a computer system based on the local area network of a design organization allowing significant time saving solutions in the design of complex construction projects at minor financial cost for construction companies. The object under design is conventionally divided into a number of design units that meet certain requirements. The data for calculating the entire structure as a whole are prepared by structural designers running the job in parallel on their PCs. The calculation of the design object as a single structure is also fulfilled on designers' PCs under a special software providing mechanical compatibility of design elements. The harmonization software algorithm is based on an iteration method. In the particular case of a static linearly-elastic problem the mechanical interactions of design units are determined by automatic generation and solution of a system of linear equations. The method proposed for constructing such computation system allows the most labor-intensive jobs like preparation of source data, design of assemblies, etc. to be run in parallel and expands the spectrum of project design tasks which can be fulfilled by a project design organization. © Nottingham University Press

关键词： distributed computer systems

来源：评论

学校读者我要写书评

暂无评论

parallel Prediction Algorithms for Heterogeneous Data: A Case Study with real-time Big Datasets

Parallel Prediction Algorithms for Heterogeneous Data: A Cas...

引用

International Conference on Big data and Cloud Computing, ICBDCC 18

作者： Lokeswari, Y.V. Jacob, Shomona Gracia Ramadoss, Rajavel Department of CSE Sri Sivasubramaniya Nadar College of Engineering Kalavakkam Chennai India Department of ECE Sri Sivasubramaniya Nadar College of Engineering Kalavakkam Chennai India

ISBN: (纸本)9789811318818

parallel data mining algorithms are extensively used to mine and discover hidden knowledge from varied, unrelated data. parallel data mining algorithms provide advantages such as reduced training time, less execution time, and less memory requirement. There are several issues in executing parallel data mining algorithms in a distributed environment. It is crucial to partition the data among processors such that there is minimal data dependency, proper synchronization, communication overhead, work load balancing among nodes in distributed processors and disk IO cost. Few of these issues can be resolved when parallel data mining algorithms are executed on Apache framework called Hadoop Map Reduce. Hadoop Map Reduce provides improved performance, reduced communication cost, reduced execution time, reduced training time, and reduced IO access. This paper proposes a novel framework that aims at enhancing the aforementioned advantages in terms of scalability by increasing the number of nodes in the Hadoop cluster and analyzing the performance of classification algorithms like K-Nearest Neighbor, Naïve Bayes and Decision Tree. This parallel framework could be extended to other fields of biotechnology where prediction on large datasets is essential. © 2019, Springer Nature Singapore Pte Ltd.

关键词： Scalability

来源：评论

学校读者我要写书评

暂无评论

The Design and Implementation of a High-Performance Portfolio Optimization Platform

The Design and Implementation of a High-Performance Portfoli...

引用

IEEE International Conference on Computational Science and Engineering, CSE

作者： Yidong Chen Zhonghua Lu Xueying Yang Computer Network Information Center Chinese Academy of Sciences University of Chinese Academic of sciences Beijing China Computer Network Information Center Chinese Academy of Sciences Beijing China

ISBN: (数字)9781665403986

ISBN: (纸本)9781665403993

Aiming at the high complexity of parameter optimization for portfolio models, this paper designs a distributed high-performance portfolio optimization platform(HPPO) based on parallel computing framework and event driven architecture. The platform consists of the data layer, the model layer, and the excursion layer, which is built in a component, pluggable, and loosely coupled way. The platform adopts parallelization acceleration for backtesting and optimizing parameters of portfolio models in a certain historical interval. The platform is able to docking portfolio model with real-time market. Based on the HPPO platform, a parallel program is designed to optimize the parameters of the value at risk(VAR) model. The performance of the platform are summarized by analyzing the experimental results and comparing with the open source framework Zipline and Rqalpha.

关键词： Adaptation models Computational modeling Computer architecture parallel processing real-time systems Optimization Portfolios

来源：评论

学校读者我要写书评

暂无评论

SCRaM – State-Consistent Replication Management for Networked Control systems

SCRaM – State-Consistent Replication Management for Network...

引用

IEEE/ACM International Conference on Cyber-Physical systems (ICCPS)

作者： Ben W. Carabelli Frank Dürr Kurt Rothermel Institute of Parallel and Distributed Systems University of Stuttgart Germany

ISBN: (数字)9781728155012

ISBN: (纸本)9781728155029

Networked control systems (NCS) consist of sensors and actuators that are connected to a controller through a packet-switched network in a feedback loop to control physical systems in diverse application areas such as industry, automotive, or power infrastructure. The control of critical real-time systems places strong requirements on the latency and reliability of both the communication network and the controller. In this paper, we consider the problem of increasing the reliability of an NCS subject to crash failures and message loss by replicating the controller component. Previous replication schemes for real-time systems have focused on ensuring that no conflicting values are sent to the actuators by different replicas. Since this property, which we call output consistency, only refers to the values within one time step, it is insufficient for reasoning about the formal conditions under which a group of replicated controllers behaves equivalent to a non-replicated controller. Therefore, we propose the stronger state consistency property, which ensures that the sequence of values produced by the replicated controller exhibits the same dynamical behaviour as a non-replicated controller. Moreover, we present SCRaM, a protocol for replicating generic periodically sampled controllers that satisfies both of these consistency requirements. To demonstrate the effectiveness of our approach, we evaluated it experimentally for the control of a cart-driven inverted pendulum.

关键词： Actuators Sensors Protocols Delays Reliability real-time systems

来源：评论

学校读者我要写书评

暂无评论

Load predictions: vulnerability of micro-grids based on renewable energies due to increasing population and individual demand

Load predictions: vulnerability of micro-grids based on rene...

引用

17th Wind and the 8th Solar Integration workshop

作者： Kuehnel, Meike Ravanbach, Babak Hanke, Benedikt Weigel, Olga Stuermer, Ingo W. McMaster, Alister Maebe, Sander von Maydell, Karsten DLR Inst Networked Energy Syst Carl von Ossietzky Str D-26129 Oldenburg Germany Deutsch Gesell Int Zusammenarbeit GIZ GmbH Hamburg Germany Lower Saxony Minist Environm Affairs Energy Bldg Hannover Germany Dept Econ Dev Environm Affairs & Tourism Eastern East London South Africa Deutsch Gesell Int Zusammenarbeit GIZ GmbH East London South Africa

Small-scale systems are vulnerable to changes. The development of solution strategies for changing conditions should, therefore, be considered from the outset in planning. For planners and installers of mini-grids for rural villages without grid connection, first of all, the question of the expected electricity demand arises. So far, this is determined primarily by interviewing the population. However, studies have shown that there are still major differences between the power demand estimated and the real load after electrification. Reinforcing here is the inadequate assessment of load development through immigration and increased prosperity. On the basis of a currently developing mini-grid in the Eastern Cape of South Africa, we could show that a load increase of similar to 60% within 10 years is to be expected. The following possibilities were outlined as possible ways of counteracting the increase in load: (i) communicating with the population, i.e. on energy efficiency to raise awareness and understanding, (ii) early planning of capacity building and identification of key performance parameters to trigger the expansion based on local socio-cultural conditions and grid supporting qualities. An adequate database for the initial electrification and power development should also be established and available on open-source basis for researchers, developers, communities, and installers.

关键词： socio-economic effects distributed power generation renewable energy sources power grids power generation planning energy conservation load predictions microgrid vulnerability renewable energies small-scale systems rural villages grid connection expected electricity demand load development minigrid Eastern Cape South Africa energy efficiency local socio-cultural conditions power development capacity building planning key performance parameter identification grid supporting qualities open-source basis time 10 0 year

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：