检索结果-内蒙古大学图书馆

Reliable Wide-Area Data Transfers for Streaming Workflows

IEEE TRANSACTIONS ON parallel AND distributed systems 2022年第12期33卷 3546-3557页

作者： Sapkota, Hemanta Arslan, Engin Univ Nevada Dept Comp Sci & Engn Reno NV 89557 USA

Many large science projects rely on remote clusters for (near) real-time data processing, thus they demand reliable wide-area data transfer performance for smooth end-to-end workflow executions. However, data transfers are often exposed to performance variations due to the changing network (e.g., background traffic) and dataset (e.g., average file size) conditions, necessitating adaptive solutions to meet stringent performance requirements of delay-sensitive streaming workflows. In this article, we propose FStream++ to provide reliable transfer performance for large streaming science applications by dynamically adjusting transfer settings to adapt to changing transfer conditions. FStream++ combines three optimization methods as dynamic tuning, online profiling, and historical analysis to swiftly and accurately discover optimal transfer settings that can meet workflow requirements. Dynamic tuning uses a heuristic model to predict the values of transfer parameters based on dataset characteristics and network settings. Since heuristic models fall short to incorporate many important factors such as I/O throughput and resource interference, we complement it with online profiling to execute a real-time search for a subset of transfer settings. Finally, historical analysis takes advantage of the long-running nature of streaming workflows by storing and analyzing previous performance observations to shorten the execution time of online profiling. We evaluate the performance of FStream++ by transferring several synthetic and real-world workloads in high-performance production networks and show that it offers up to 3:6x performance improvement over legacy transfer applications and up to 24% over our previous work FStream.

关键词： distributed workflows high-speed networks online profiling streaming science applications throughput optimization

来源：评论

学校读者我要写书评

暂无评论

Lightweight Streaming Graph Partitioning by Fully Utilizing Knowledge from Local View 43

Lightweight Streaming Graph Partitioning by Fully Utilizing ...

引用

43rd IEEE International Conference on distributed Computing systems (ICDCS)

作者： Wang, Zhigang Yang, Zichao Wang, Ning Du, Yujie Nie, Jie Wei, Zhiqiang Gu, Yu Yu, Ge Ocean Univ China Qingdao Peoples R China Yantai Engn & Tech Coll Yantai Peoples R China Northeastern Univ Boston MA 02115 USA

ISBN: (纸本)9798350339864

Data partitioning is the most fundamental procedure before parallelizing complex analysis on very big graphs. As a classical NP-complete problem, graph partitioning usually employs offline or online/streaming heuristics to find approximately optimal solutions. However, they are either heavyweight in space and time overheads or suboptimal in quality measured by workload balance and the number of cutting edges across partitions, both of which cannot scale well with the ever-growing demands of quickly analyzing big graphs. This paper thereby proposes a new vertex partitioner for better scalability. It preserves the lightweight advantage of existing streaming heuristics, and more importantly, fully utilizes the knowledge embedded in the local view when streaming a vertex, which significantly improves the quality. We present a sliding window technique to compensate for the additional memory costs caused by knowledge utilization. Also, a parallel technique with dependency detection optimization is designed to further enhance efficiency. Experiments on a spread of real-world datasets validate that our proposals can achieve overall success in terms of partitioning quality, memory consumption, and runtime efficiency.

关键词： Streaming Graph Partitioning Local Streaming View Memory Consumption parallel Partitioning

来源：评论

学校读者我要写书评

暂无评论

Safety Enhancement for real-time parallel Applications in distributed Automotive Embedded systems: A Stable Stopping Approach

引用

IEEE TRANSACTIONS ON parallel AND distributed systems 2020年第9期31卷 2067-2080页

作者： Xie, Guoqi Zeng, Gang Li, Renfa Hunan Univ Coll Comp Sci & Elect Engn Key Lab Embedded & Cyber Phys Syst Hunan Prov Changsha 410082 Hunan Peoples R China Nagoya Univ Grad Sch Engn Nagoya Aichi 4648603 Japan

In distributed automotive embedded systems, safety issues run through the entire life cycle, and safety mechanisms for error handling are desirable for risk control. This article focuses on safety enhancement (i.e., safety mechanisms for error handling) for a safety-critical automotive application within its deadline. A stable stopping approach used for safety enhancement for an automotive application is proposed based on the static recovery mechanism provided in ISO 26262. The Stable Stopping-based Safety Enhancement (SSSE) approach is proposed by combining known backward recovery, proposed forward recovery, and proposed forward-and-backward recovery through primary-backup repetition. The stable stopping (i.e., SSSE) approach is a convergence algorithm, which means that when the reliability value reaches a steady state and the algorithm can stop. Experimental results reveal that the exposure level defined in ISO 26262 drops from E3 to E1 after using SSSE, and such improvement enables a safety guarantee of higher level.

关键词： distributed automotive embedded systems safety enhancement stable stopping

来源：评论

学校读者我要写书评

暂无评论

Communication Interchange For Artificial Intelligence systems

Communication Interchange For Artificial Intelligence System...

引用

International Conference on Computing, Networking and Communications (ICNC)

作者： Voicu, Razvan Cristian Pande, Aarush Kaunteya Tanveer, Muhammad Hassan Chang, Yusun Kennesaw State Univ Dept Robot & Mechatron Engn Atlanta GA 30339 USA

ISBN: (纸本)9798350371000;9798350370997

The rise and proliferation of Artificial Intelligence (AI) technologies are bringing transformative changes to various sectors, signaling a new era of innovation in fields as diverse as medicine, manufacturing, and even day-to-day social interactions. Notable advancements are not just confined to textual understanding, as seen in models like GPT, but also extend to visual cognition through image recognition and more. Beyond surface interactions and predictions, AI finds profound applications in life-saving domains such as medical diagnostics and becomes an integral part of daily life through chatbot-based customer interactions. However, as the horizon of AI expands, a crucial yet often overlooked aspect emerges- the underlying mission-critical infrastructure required to support and deploy these models effectively. The intricacies of efficient communication systems, foundational for real-time AI model operations, take center stage in ensuring the seamless functioning of AI-driven applications. This paper explores the quintessential changes needed in communication paradigms to keep pace with the evolving AI landscape. Specifically, we highlight the pivotal role of multipath communication in enhancing the responsiveness and efficiency of AI applications [1]. As a case in point, we investigate its impact on mission-critical operations in robotics. Through experimentation and analysis, the results elucidate the substantial benefits of this approach, revealing a significant improvement in delay metrics. This work underscores the imperative of aligning communication systems with the ever-growing demands of AI, ensuring that infrastructural capabilities do not lag in the race for innovation.

关键词： Artificial Intelligence (AI) Programmable Networking parallel Communication Centralized - distributed systems real-time Control Multipath Communication AI Infrastructure

来源：评论

学校读者我要写书评

暂无评论

Utilizing Blockchain Technology for Financial Services in parallel, distributed, and Grid Computing Frameworks

Utilizing Blockchain Technology for Financial Services in Pa...

引用

2024 International Conference on Computing and Intelligent reality Technologies, ICCIRT 2024

作者： Chokkamreddy, Prakash Palagati, Anusha Rao, Akurathi Lakshmi Pathi Maheswari, P. Mahadevan, Vanitha Balan, Santhosh Kumar School of Management Studies Guru Nanak Institutions Technical Campus Ibrahimpatnam Hyderabad India Dept. of Computer Science and Engineering Guru Nanak Institute of Technology Ibrahimpatnam Hyderabad India Department of Computer Applications KSR College of Arts and Science for Women Tiruchengode India College of Engineering and Technology University of Technology and Applied Sciences Al Mussanah Oman

ISBN: (纸本)9798331510299

Application of blockchain in financial services has opened new ways of efficiency in transaction processing, assets management and security. The application of parallel, distributed, and grid computing with blockchain in improving the functionality of financial services like real time financial transactions, high frequency trading, portfolio and risk analysis is examined in this paper. It has been ascertained that namely parallel and distributed computing together with blockchain are the most suitable for facing increasing needs in data in the sphere of financial services. It also investigates the use of grid computing frameworks and their effects on the correspondingly balanced flows of medium amongst multiple systems. Various issues such as security, privacy violations, and adherence to the law are also reviewed, and existing prospects for developing through smart contracts and consensus mechanisms using blockchain. Exploring how some of the most successful pilot use-cases of blockchain are applicable for future growth in the financial services industry offers insights into how blockchain will soon enter mainstream as a disruptive technology in the broader global financial sector. © 2024 IEEE.

关键词： Smart contract

来源：评论

学校读者我要写书评

暂无评论

parallel and distributed Computing for Anomaly Detection From Hyperspectral Remote Sensing Imagery

引用

PROCEEDINGS OF THE IEEE 2021年第8期109卷 1306-1319页

作者： Du, Qian Tang, Bo Xie, Weiying Li, Wei Mississippi State Univ Dept Elect & Comp Engn Mississippi State MS 39762 USA Xidian Univ State Key Lab Integrated Serv Networks Xian 710071 Shaanxi Peoples R China Beijing Inst Technol Sch Informat & Elect Beijing 100081 Peoples R China

Anomaly detection from remote sensing images is to detect pixels whose spectral signatures are different from their background. Anomalies are often man-made targets. With such target signatures being unknown, anomaly detection has many important applications, such as water quality monitoring, crop stress surveying, and law enforcement-related uses, where prior information of targets is often unavailable. The key to success is accurate background modeling. Anomaly detection from remote sensing images is challenging because spatial coverage is very large and the background is highly heterogeneous. For pixel-based anomaly detection, computing cost in background modeling and a spatial-convolution-type detection process is very expensive. Thus, parallel and distributed computing is critical in reducing execution time, which can fit the need for real-time or near real-time detection from airborne and spaceborne platforms in support of immediate decision-making. This article reviews the recent advances in anomaly detection from hyperspectral remote sensing images and their implementation using parallel and distributed systems. The classical methods, i.e., the Reed-Xiaoli (RX) algorithm and its variants, including its real-time processing version, are illustrated in commodity graphic processing units (GPUs), cloud, and field-programmable gate array (FPGA) implementations. Practical issues and future development trends are also discussed.

关键词： Anomaly detection Hyperspectral imaging Feature extraction real-time systems distributed databases Detectors Principal component analysis Anomaly detection distributed computing field-programmable gate array (FPGA) hyperspectral remote sensing large-scale data parallel computing real-time processing

来源：评论

学校读者我要写书评

暂无评论

Near-Linear Scaling Data parallel Training with Overlapping-Aware Gradient Compression 29

Near-Linear Scaling Data Parallel Training with Overlapping-...

引用

29th IEEE International Conference on parallel and distributed systems, ICPADS 2023

作者： Meng, Lin Sun, Yuzhong Li, Weimin Chinese Academy of Sciences Institute of Computing Technology China University of Chinese Academy of Sciences China

ISBN: (纸本)9798350330717

Existing Data parallel (DP) trainings for deep neural networks (DNNs) often experience limited scalability in speedup due to substantial communication overheads. While Overlapping technique can mitigate such problem by paralleling communication and computation in DP, its effectiveness is constrained by the high communication-to-computation ratios (CCR) of DP training tasks. Gradient compression (GC) is a promising technique to obtain lower CCR by reducing communication volume directly. However, it is challenging to obtain real performance improvement by applying GC into Overlapping because of (1) severe performance penalties in traditional GCs caused by high compression overhead and (2) decline of Overlapping benefit owing to the possible data dependency in GC schemes. In this paper, we propose COVAP, a novel GC scheme designing a new coarse-grained filter, makes the compression overhead close to zero. COVAP ensures an almost complete overlap of communication and computation by employing adaptive compression ratios and tensor sharding tailored to specific training tasks. COVAP also adopts an improved error feedback mechanism to maintain training accuracy. Experiments are conducted on Alibaba Cloud ECS instances with different DNNs of real-world applications. The results illustrate that COVAP outperforms existent GC schemes in time-to-solution by 1.92x-15.39x and exhibits near-linear scaling. Furthermore, COVAP achieves best scalability under experiments on four different cluster sizes. © 2023 IEEE.

关键词： data parallel distributed deep learning gradient compression Overlapping

来源：评论

学校读者我要写书评

暂无评论

parallel Islands: A parallel Computing Educational Video Game 2024

Parallel Islands: A Parallel Computing Educational Video Gam...

引用

55th ACM Technical Symposium on Computer Science Education, SIGCSE 2024

作者： Cameron, Melissa Ellis, Margaret Nikolopoulos, Dimitrios Virginia Tech BlacksburgVA United States

ISBN: (纸本)9798400704246

The rise in multiprocessors has led to the incorporation of parallel processing in virtually all segments of industry. Creation of and maintenance for the software to run these systems, as well as for the applications using these systems, requires extensive knowledge of the concepts and skills of parallel and distributed computing (PDC). This will naturally lead to an increase in the demand for software developers familiar with PDC and an increase in the demand for universities to incorporate PDC concepts into all levels of their curricula including introductory courses. Because there are real and perceived difficulties in teaching PDC concepts, particularly early in the CS curriculum there is a need to produce educational materials to assist with this expansion. At the same time CS education is wrestling with the surge in the need for graduates with PDC skills, it is also attempting to overcome a gender imbalance in CS. The necessity to create the materials required for increasing PDC education provides an opportunity to make strides in increasing gender diversity in CS as well. Therefore, parallel Islands was created as a tool to aid in introducing PDC concepts in introductory CS courses in a manner that appeals to all gender identities. It is a web-based educational video game;along with the associated pre- and post-gameplay introductory, review and discussion sections, that is designed to be used by students autonomously or with instructor oversight. © 2024 Owner/Author.

关键词： Curricula

来源：评论

学校读者我要写书评

暂无评论

Comparison of parallel B Static Traffic Assignment Algorithm Implementation in Java and C++

Comparison of Parallel B Static Traffic Assignment Algorithm...

引用

International workshop/Symposium on distributed Simulation and real-time Applications (DS-RT)

作者： Tomas Potuzak Petr Pernicka Frantisek Kolovsky Department of Computer Science and Engineering Faculty of Applied Sciences University of West Bohemia Plzen Czech Republic RoadTwin S.R.O Plzen Czech Republic

ISBN: (数字)9798331527211

ISBN: (纸本)9798331527228

This paper compares sequential and parallel Java and C++ implementations of the B algorithm, a relatively new algorithm for user-equilibrium (UE) (road) traffic assignment (TA). All the versions were implemented in an optimized way while using best practices in both languages. Their performances were thoroughly tested using multiple large real road traffic networks. The tests showed that the C++ version is indeed faster in most cases, but only by 8.54 % on average.

关键词： Java C++ languages Road traffic real-time systems Best practices

来源：评论

学校读者我要写书评

暂无评论

Improving real-time Service Quality Through parallel Monte Carlo Tree Search

Improving Real-Time Service Quality Through Parallel Monte C...

引用

International workshop on Quality of Service

作者： Jiajia Zhang Shuman Zhuang Yulin Wu School of Computer Science Harbin Institite of Technology(ShenZhen) Shenzhen China

ISBN: (数字)9798350350128

ISBN: (纸本)9798350350135

In this paper, we present a new algorithm named distributed Multi-root Pipeline MCTS (DMP-MCTS), to improve real-time search efficiency in multi-machine scenarios. By utilizing root parallel technology with significance detection and pipeline pattern for parallel MCTS (3PMCTS), we not only reduce the repetitive computing tasks among subtrees but also allow for flexible operation time based resource allocation. The experiment shows this work achieves better computational performance under linear acceleration conditions, compared with other existing works.

关键词： Monte Carlo methods Pipelines Quality of service parallel processing real-time systems Resource management

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：