检索结果-内蒙古大学图书馆

A Stepwise Auto-Profiling Method for Performance Optimization of Streaming Applications

ACM TRANSACTIONS ON AUTONOMOUS AND ADAPTIVE systems 2018年第4期12卷 1–33页

作者： Liu, Xunyun Dastjerdi, Amir Vahid Calheiros, Rodrigo N. Qu, Chenhao Buya, Rajkumar Univ Melbourne Sch Comp & Informat Syst Cloud Comp & Distributed Syst CLOUDS Lab Melbourne Vic Australia PwC Australia Sydney NSW Australia Western Sydney Univ Penrith NSW Australia 2 Riverside Quay Southbank Vic 3006 Australia Locked Bag 1797 Penrith NSW 2751 Australia

Data stream management systems (DSMSs) are scalable, highly available, and fault-tolerant systems that aggregate and analyze real-time data in motion. To continuously perform analytics on the fly within the stream, state-of-the-art DSMSs host streaming applications as a set of interconnected operators, with each operator encapsulating the semantic of a specific operation. For parallel execution on a particular platform, these operators need to be appropriately replicated in multiple instances that split and process the workload simultaneously. Because the way operators are partitioned affects the resulting performance of streaming applications, it is essential for DSMSs to have a method to compare different operators and make holistic replication decisions to avoid performance bottlenecks and resource wastage. To this end, we propose a step-wise profiling approach to optimize application performance on a given execution platform. It automatically scales distributed computations over streams based on application features and processing power of provisioned resources and builds the relationship between provisioned resources and application performance metrics to evaluate the efficiency of the resulting configuration. Experimental results confirm that the proposed approach successfully fulfills its goals with minimal profiling overhead.

关键词： Stream processing data stream management systems performance optimization resource management

来源：评论

学校读者我要写书评

暂无评论

Efficient distributed All-Pairs Algorithms: Management Using Optimal Cyclic Quorums

引用

IEEE TRANSACTIONS ON parallel AND distributed systems 2018年第2期29卷 391-404页

作者： Kleinheksel, Cory James Somani, Arun K. Iowa State Univ Dept Elect & Comp Engn Ames IA 50011 USA

All Pairs problems occur in many research fields. The all-pairs problem requires all data elements to be paired with all other data elements. With the advent of new data intensive big data applications and increase in data size, methods to reduce memory foot print and distribute to work equally across compute nodes are needed. In this paper, we propose cyclic quorum sets for all-pairs algorithm computations to reduce memory foot print. We show that the cyclic quorum sets have a unique all-pairs property that allows for minimal data replication. The cyclic quorums set based computing requires only N/root P size memory, up to 50 percent smaller than the dual N/root P N/root P array implementations proposed earlier, and significantly smaller than solutions requiring all data in each node. Computation can be distributed efficiently and more importantly and are communication-less after initial data distribution, which is a huge advantage in minimizing computation time. Scaling from 16 to 512 cores ( 1 to 32 compute nodes), our application experiments on a real dataset demonstrated scalability with greater than 150x (super-linear) speedup with less than 1/4th the memory usage per node in our experiments.

关键词： distributed all-pair computations cyclic quorum set and all pair property communication-less computation memory foot print efficiency

来源：评论

学校读者我要写书评

暂无评论

Performance Implications of Global Virtual time Algorithms on a Knights Landing Processor 18

Performance Implications of Global Virtual Time Algorithms o...

引用

22nd IEEE/ACM International Symposium on distributed Simulation and real time Applications (DS-RT)

作者： Eker, Ali Williams, Barry Mishra, Nitesh Thakur, Dushyant Chiu, Kenneth Ponomarev, Dmitry Abu-Ghazaleh, Nael SUNY Binghamton Binghamton NY 13901 USA Univ Calif Riverside Riverside CA 92521 USA

ISBN: (纸本)9781538650486

Recent studies investigated the performance of parallel Discrete Event Simulation (PDES) on Intel Xeon Phi many-core processors, but generally reported underwhelming performance results, especially at high scales when all cores and thread contexts are fully loaded. While the lack of scalability in an earlier study on a Knights Corner (KC) processor is an artifact of physical limitations of the KC system, performance challenges on a Knights Landing (KNL) system partially stem from a slower global virtual time (GVT) computation algorithm used in that study. In this paper, we re-examine PDES performance on KNL under more efficient GVT algorithms to alleviate the GVT bottleneck. Specifically, we compare a synchronous GVT algorithm based on barrier synchronization, and two asynchronous GVT implementations: a modified Mattern's algorithm for shared memory systems and a recently-proposed wait-free algorithm. Using the ROSS simulator, we demonstrate that minimizing the GVT bottleneck results in significant improvement in scalability, allowing the simulation to scale with performance all the way to 250 threads (per chip). Interestingly, we observe that while for the balanced models the wait-free algorithm is a clear winner, barrier-based GVT provides significantly better results for imbalanced models executed at high scale. We also perform detailed simulation profiling to understand the underlying reasons for these performance trends.

关键词： parallel Discrete Event Simulation Intel Xeon Phi Knights Landing Manycore Architectures Performance Global Virtual time

来源：评论

学校读者我要写书评

暂无评论

parallel Active Object Pattern: Extension of the Active Object Pattern through parallel Programming Design Patterns 31

Parallel Active Object Pattern: Extension of the Active Obje...

引用

31st GI/ITG International Conference on Architecture of Computing systems, ARCS 2018

作者： Schmid, Michael Langer, Tobias Osinski, Lukas Mottok, Jürgen Laboratory for Safe and Secure Systems-LaS3 University of Applied Sciences Regensburg Germany

ISBN: (纸本)9783800745593

With parallel applications becoming more and more popular even in real-time systems, the demand for safe and easy-to-use software libraries and frameworks for parallel and con-current computations is growing immensely. These frameworks usually provide an implementation for different sets of software patterns. A very well known software pattern for concurrency is the Active Object pattern, that allows various threads to have synchronized access to an object in question. This paper presents the parallel Active Object pattern, which extends the common Active Object pattern to support the use of objects, whose computations are profoundly enhanced by a parallel execution. Furthermore, a C++ software framework is introduced, which implements the parallel Active Object pattern and thus provides the possibility of using task or data parallel patterns, for example Map, Reduce and Divide-and-Conquer, on the active object's calculations. The proposed framework is evaluated against two other popular libraries, namely OpenMP and Intel Threading Building Blocks. Through utilization of the C++11 standard and template classes a simple user interface is provided, which abstracts the distribution of workloads among the worker threads. By making use of the C++ Standard Template Library the framework can easily be ported to embedded systems and by extending the pattern through real-time capabilities, which ensure a timely and reliable execution of the method requests, the intention of providing the framework for time critical environments is also targeted in the future. © ARCS 2018.

关键词： real time systems

来源：评论

学校读者我要写书评

暂无评论

SkyBridge: Fast and Secure Inter-Process Communication for Microkernels 19

SkyBridge: Fast and Secure Inter-Process Communication for M...

引用

14th EuroSys Conference

作者： Mi, Zeyu Li, Dingji Yang, Zihan Wang, Xinran Chen, Haibo Shanghai Jiao Tong Univ Inst Parallel & Distributed Syst Shanghai Key Lab Scalable Comp Syst Shanghai Peoples R China

ISBN: (纸本)9781450362818

Microkernels have been extensively studied over decades. However, IPC (Inter-Process Communication) is still a major factor of run-time overhead, where fine-grained isolation usually leads to excessive IPCs. The main overhead of IPC comes from the involvement of the kernel, which includes the direct cost of mode switches and address space changes, as well as indirect cost due to the pollution of processor structures. In this paper, we present SkyBridge, a new communication facility designed and optimized for synchronous IPC in microkernels. SkyBridge requires no involvement of kernels during communication and allows a process to directly switch to the virtual address space of the target process and invoke the target function. SkyBridge retains the traditional virtual address space isolation and thus can be easily integrated into existing microkernels. The key idea of SkyBridge is to leverage a commodity hardware feature for virtualization (i.e., VMFUNC) to achieve efficient IPC. To leverage the hardware feature, SkyBridge inserts a tiny virtualization layer (Rootkernel) beneath the original microkernel (Subkernel). The Rootkernel is carefully designed to eliminate most virtualization overheads. SkyBridge also integrates a series of techniques to guarantee the security properties of IPC. We have implemented SkyBridge on three popular opensource microkernels (seL4, Fiasco. OC, and Google Zircon). The evaluation results show that SkyBridge improves the speed of IPC by 1.49x to 19.6x for microbenchmarks. For real-world applications (e.g., SQLite3 database), SkyBridge improves the throughput by 81.9%, 1.44x and 9.59x for the three microkernels on average.

关键词： Virtualization

来源：评论

学校读者我要写书评

暂无评论

distributed Louvain Algorithm for Graph Community Detection 32

Distributed Louvain Algorithm for Graph Community Detection

引用

27th International Heterogeneity in Computing workshop in conjunction with 32nd IEEE International parallel and distributed Processing Symposium (IPDPS)

作者： Ghosh, Sayan Halappanavar, Mahantesh Tumeo, Antonino Kalyanaraman, Ananth Lu, Hao Chavarria-Miranda, Daniel Khan, Arif Gebremedhin, Assefaw H. Washington State Univ Pullman WA 99164 USA Pacific Northwest Natl Lab Richland WA USA Trovares Inc Seattle WA USA Oak Ridge Natl Lab Oak Ridge TN USA

ISBN: (纸本)9781538643686

In most real-world networks, the nodes/vertices tend to be organized into tightly-knit modules known as communities or clusters, such that nodes within a community are more likely to be "related" to one another than they are to the rest of the network. The goodness of partitioning into communities is typically measured using a well known measure called modularity. However, modularity optimization is an NP-complete problem. In 2008, Blondel, et al. introduced a multi-phase, iterative heuristic for modularity optimization, called the Louvain method. Owing to its speed and ability to yield high quality communities, the Louvain method continues to be one of the most widely used tools for serial community detection. In this paper, we present the design of a distributed memory implementation of the Louvain algorithm for parallel community detection. Our approach begins with an arbitrarily partitioned distributed graph input, and employs several heuristics to speedup the computation of the different steps of the Louvain algorithm. We evaluate our implementation and its different variants using real-world networks from various application domains (including internet, biology, social networks). Our MPI+ OpenMP implementation yields about 7x speedup (on 4K processes) for soc-friendster network (1.8B edges) over a state-of-the-art shared memory multicore implementation (on 64 threads), without compromising output quality. Furthermore, our distributed implementation was able to process a larger graph (uk-2007;3.3B edges) in 32 seconds on 1K cores (64 nodes) of NERSC Cori, when the state-of-the-art shared memory implementation failed to run due to insufficient memory on a single Cori node containing 128 GB of memory.

关键词： Community detection parallel Louvain distributed graph clustering parallel graph heuristics

来源：评论

学校读者我要写书评

暂无评论

MDDRSPF: A Model Driven distributed real-time Stream Processing Framework

MDDRSPF: A Model Driven Distributed Real-Time Stream Process...

引用

IEEE International Conference on Big Data and Cloud Computing (BdCloud)

作者： Yijun Wen Li Zhang Cheng Wang School of Software Tsinghua University & Naval Research Academy (NVRA) Beijing China School of Software Tsinghua University Beijing China

ISBN: (数字)9781728143286

ISBN: (纸本)9781728143293

Some enterprises have demands for distributed stream processing framework to manage and process diverse business services. But their business is usually user-oriented, the access data sources and outputs are heterogeneous and need to be configurable, and the user-oriented service mode requires the platform have the ability to deploy new algorithm dynamically and provide on-demand services. As we know, there is no system that can meet all these requirements, hence a Model Driven distributed real-time Stream Processing Framework is proposed. By building project and task models, the functional model is separated from the implementation platform. The framework support multi-source heterogeneous data input and configurable output, on-demand services based on user's request, and algorithm dynamic loading. Service oriented architecture is deployed to lower the threshold of algorithm development and deployment. Furthermore, the functionality and performance is evaluated with a practical case to verify whether the system can meet the needs of practical application.

关键词： Task analysis Data models real-time systems Load modeling Computational modeling Heuristic algorithms

来源：评论

学校读者我要写书评

暂无评论

Skitter: A DSL for distributed Reactive Workflows 5

Skitter: A DSL for Distributed Reactive Workflows

引用

5th ACM SIGPLAN International workshop on Reactive and Event-Based Languages and systems (REBLS)

作者： Saey, Mathijs De Koster, Joeri De Meuter, Wolfgang Vrije Univ Brussel Software Languages Lab Brussels Belgium

ISBN: (纸本)9781450360708

Writing real-time applications that react to vast amounts of incoming data is a hard problem, as the volume of incoming data implies the need for distributed execution on a cluster architecture. We envision such an application can be created as a data processing pipeline which consists of a set of generic, reactive components, which may be reused in other applications. However, there is currently no programming model or framework that enables the reactive, scalable execution of such a pipeline on a cluster architecture. Our work introduces the notion of reactive workflows, a technique that combines concepts from scientific workflows and reactive programming. Reactive workflows enable the integration of these generic components into a single workflow that can be executed on a cluster architecture in a reactive, scalable way. To deploy these reactive workflows, we introduce a domain specific language, called Skitter. Skitter enables developers to write reactive components and compose these into reactive workflows, which can be distributed over a cluster by Skitter's runtime system.

关键词： Scientific Workflows distributed Programming Reactive Programming

来源：评论

学校读者我要写书评

暂无评论

A Joint real time Optimization of Household Loads, Energy Storage and Peak Generator for Stand-Alone distributed PV systems

A Joint Real Time Optimization of Household Loads, Energy St...

引用

IEEE International Conference on Communications (ICC) / workshop on Integrating UAVs into 5G

作者： Ahmad, A. Khan, Jamil Y. Univ Newcastle Sch Elect Engn & Comp Callaghan NSW 2308 Australia

ISBN: (纸本)9781538631805

We investigate a real time (RT) communication and control technique for a user centric stand-alone distributed photovoltaic (PV) system. The current user centric RT techniques for grid-connected systems may not be directly employed in stand-alone systems due to contradiction in their objectives. In this paper, we propose a user centric RT control algorithm for stand-alone PV systems and investigate its communication requirements. Our aim is to minimize an average aggregate system cost considering a joint optimization problem of load scheduling, energy procurement from a peak generator (PG) and battery energy storage (BES) management. By subsequent relaxation and reformulation of the problem, we employ the Lyapunov optimization technique (LOT). Simulation results show the effectiveness of the proposed RT solution in terms of the selected performance metrics.

关键词： Communication Energy storage load scheduling peak generator PV stand-alone RT optimization

来源：评论

学校读者我要写书评

暂无评论

Implicit Decomposition for Write-Efficient Connectivity Algorithms 32

Implicit Decomposition for Write-Efficient Connectivity Algo...

引用

27th International Heterogeneity in Computing workshop in conjunction with 32nd IEEE International parallel and distributed Processing Symposium (IPDPS)

作者： Ben-David, Naama Blelloch, Guy E. Fineman, Jeremy T. Gibbons, Phillip B. Gu, Yan McGuffey, Charles Shun, Julian Carnegie Mellon Univ Pittsburgh PA 15213 USA Georgetown Univ Washington DC 20057 USA MIT CSAIL Cambridge MA 02139 USA

ISBN: (纸本)9781538643686

The future of main memory appears to lie in the direction of new technologies that provide strong capacity-toperformance ratios, but have write operations that are much more expensive than reads in terms of latency, bandwidth, and energy. Motivated by this trend, we propose sequential and parallel algorithms to solve graph connectivity problems using significantly fewer writes than conventional algorithms. Our primary algorithmic tool is the construction of an o(n)-sized implicit decomposition of a bounded-degree graph G on n nodes, which combined with read-only access to G enables fast answers to connectivity and biconnectivity queries on G. The construction breaks the linear-write "barrier", resulting in costs that are asymptotically lower than conventional algorithms while adding only a modest cost to querying time. For general non-sparse graphs on m edges, we also provide the first parallel algorithms for connectivity and biconnectivity that require o(m) writes and O(m) operations. These algorithms provide insight into how applications can efficiently process computations on large graphs in systems with read-write asymmetry.

关键词： Random access memory Computational modeling Task analysis parallel algorithms Writing Standards Clustering algorithms"

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：