检索结果-内蒙古大学图书馆

IEEE international conference on Big Data and Cloud Computing (BdCloud)

作者： Yidong Zhong Yang Ao Dingding Li Jianguo Li Deze Zeng School of Computer Science South China Normal University Guangzhou Guangdong China School of Computer Science China University of Geosciences Wuhan Hubei China

In traditional CPU scheduling systems, it is challenging to customize scheduling policies for datacenter workloads. Therefore, distributed cluster managers can only perform coarse-grained job scheduling rather than fine-grained CPU scheduling, although the latter is also critical to application performance in the clusters. Some scheduling systems have been proposed to facilitate the implementation and deployment of custom policies. However, these systems neither adequately capture the resource requirements of workloads nor cooperatively manage the CPU scheduling of distributed nodes, thus may failing to fully utilize CPU resources in the clusters. To tackle these issues, we present ADSwitch, a distributed CPU scheduling system that manages CPU resources adaptively in datacenter clusters. We train an efficient workload classification model based on a recurrent neural network (RNN), which enables ADSwitch to dynamically identify the demand of the running workload and switch to an appropriate scheduling policy through a reliable online update mechanism. We implement ADSwitch and evaluate it in a distributed cluster with synthetic workloads. The experimental results demonstrate that ADSwitch achieves up to 33% lower tail latency and 28% higher throughput than the existing scheduling systems without workload characteristic awareness.

关键词：

来源：评论

学校读者我要写书评

暂无评论

Clock Synchronization Protocols and Models for Wireless Sensor Networks

Clock Synchronization Protocols and Models for Wireless Sens...

引用

international conference on Computational Intelligence and Communication Networks (CICN)

作者： Geetha S Latha P A Raajya Vardhini Suganya R M.S. Nidhya V.S.B Engineering College Bangalore India School of Computer Applications Dayananda Sagar University Bangalore

ISBN: (数字)9798331505264

ISBN: (纸本)9798331505271

Clock synchronization is a critical aspect in the operation of Wireless Sensor Networks (WSNs), ensuring that sensor nodes maintain accurate time for coordinated communication and data processing. Various clock synchronization algorithms have been developed to address the synchronization challenges in WSNs, aiming to align the clocks of distributed nodes efficiently. These algorithms play a crucial role in achieving synchronization by compensating for clock drift, which can significantly impact the accuracy of timekeeping among sensor nodes. Timing constraints are fundamental in WSNs, as they influence network performance and data transmission reliability. Addressing timing constraints is essential for optimizing network operations and ensuring timely data delivery. In this paper we analysed various synchronization protocols, Algorithm and techniques. Finally, we compared features of TPSN and RBS protocols, based on accuracy, robustness TPSN is best one.

关键词： Wireless sensor networks Accuracy Protocols Scalability Data processing Robustness Energy efficiency Synchronization Data communication Clocks

来源：评论

学校读者我要写书评

暂无评论

Multi-Agent Reinforcement Learning based distributed Renewable Energy Matching for Datacenters 21

Multi-Agent Reinforcement Learning based Distributed Renewab...

引用

50th international conference on parallel processing (ICPP)

作者： Wang, Haoyu Shen, Haiying Gao, Jiechao Zheng, Kevin Li, Xiaoying Univ Virginia Charlottesville VA 22901 USA

ISBN: (纸本)9781450390682

The rapid growth of cloud computing in cloud datacenters in recent decades greatly increases the brown energy consumption in datacenters, and hence significant increase of carbon emission that negatively impacts on the environment as well as the monetary cost. More and more cloud service providers are adopting renewable energy as the energy supply to offset the consumption of brown energy. Meanwhile, an increasing number of renewable energy generators have been built to meet the needs. However, the instability of the renewable energy cannot guarantee the support to the datacenter and the energy competition of different datacenters may lead to datacenter energy outage. In this paper, we focus on the problem of how to match different renewable energy generators to the datacenters from different cloud providers to minimize the carbon emission, monetary cost, and service level objective (SLO) violation due to renewable energy shortage. The challenges here are that the datacenters may compete in energy requesting, the renewable energy generation is not stable and the decision should be made quickly. There have been no previous efforts devoting to this problem. To solve the problem, we first test several machine learning techniques on long-term prediction accuracy on renewable energy generation and energy demand using real traces and identify SARIMA for the prediction. We then propose a multi-agent reinforcement learning based method (MARL) for each datacenter to determine how much renewable energy to request from each generator based on the predicted results. We also propose a deadline guaranteed job postponement method (DGJP) to postpone executing unurgent jobs upon insufficient renewable energy supply. The trace-driven experiments show that MARL outperforms other methods by increasing up to 35% SLO satisfaction ratio, and reducing up to 19% (0.33 billion dollars in 90 days) total monetary cost and 33% total carbon emission, and DGJP further improves the performance.

关键词： Renewable energy resources

来源：评论

学校读者我要写书评

暂无评论

Optimizing distributed Training on Frontier for Large Language Models

Optimizing Distributed Training on Frontier for Large Langua...

引用

ISC High Performance 2024 Research Paper Proceedings (39th international conference)

作者： Sajal Dash Isaac R Lyngaas Junqi Yin Xiao Wang Romain Egele J. Austin Ellis Matthias Maiterth Guojing Cong Feiyi Wang Prasanna Balaprakash Oak Ridge National Laboratory Université Paris-Saclay AMD

Large language models (LLMs) have demonstrated remarkable success as foundational models, benefiting various downstream applications through fine-tuning. Loss scaling studies have demonstrated the superior performance of larger LLMs compared to their smaller counterparts. Nevertheless, training LLMs with billions of parameters poses significant challenges and requires considerable computational resources. For example, training a one trillion parameter GPT-style model on 20 trillion tokens requires a staggering 120 million exaflops. This research explores efficient distributed training strategies to extract this computation from Frontier, the world's first exascale supercomputer. We enable and investigate various model and data parallel training techniques, such as tensor parallelism, pipeline parallelism, and sharded data parallelism, to facilitate training a trillion-parameter model on Frontier. We empirically assess these techniques and their associated parameters to determine their impact on memory footprint, communication latency, and GPU's computational efficiency. We analyze the complex interplay among these techniques and find a strategy to combine them to achieve high throughput through hyperparameter tuning. We have identified efficient strategies for training large LLMs of varying sizes through empirical analysis and hyperparameter tuning. For 22 Billion, 175 Billion, and 1 Trillion parameters, we achieved GPU throughputs of 38.38%, 36.14%, and 31.96%, respectively. For the training of the 175 Billion parameter model and the 1 Trillion parameter model, we achieved 100% weak scaling efficiency on 1024 and 3072 Mi250X GPUs, respectively. We also achieved strong scaling efficiencies of 89% and 87% for these two models. We trained these models only tens of iterations instead of training till completion.

关键词：

来源：评论

学校读者我要写书评

暂无评论

Efficient processing of Streaming Data using Multiple Abstractions 14

Efficient Processing of Streaming Data using Multiple Abstra...

引用

IEEE 14th international conference on Cloud Computing (CLOUD)

作者： Qadeer, Abdul Heidemann, John Univ Southern Calif Informat Sci Inst Los Angeles CA 90007 USA

ISBN: (纸本)9781665400602

Large websites and distributed systems employ sophisticated analytics to evaluate successes to celebrate and problems to be addressed. As analytics grow, different teams often require different frameworks, with dozens of packages supporting with streaming and batch processing, SQL and no-SQL. Bringing multiple frameworks to bear on a large, changing dataset often create challenges where data transitions-these impedance mismatches can create brittle glue logic and performance problems that consume developer time. We propose Plumb, a meta-framework that can bridge three different abstractions to meet the needs of a large class of applications in a common workflow. Large-block streaming (Block-Streaming) is suitable for single-pass applications that care about the temporal and spatial locality. Windowed-Streaming allows applications to process a group of data and many reductions. Stateful-Streaming enables applications to keep a long-term state and always-on behavior. We show that it is possible to bridge abstractions, with a common, high-level workflow specification, while the system transitions data batch processing and block- and record-level streaming as required. The challenge in bridging abstractions is to minimize latency while allowing applications to select between sequential and parallel operation, while handling out-of-order data delivery, component failures, and providing clear semantics in the face of missing data. We demonstrate these abstractions evaluating a 10-stage workflow of DNS analytics that has been in production use with Plumb for 2 years, comparing to a brittle hand-built system that has run for more than 3 years.

关键词： programming model abstraction big-data abstraction large-block streaming programmer productivity

来源：评论

学校读者我要写书评

暂无评论

Efficient Asynchronous GCN Training on a GPU Cluster 27

Efficient Asynchronous GCN Training on a GPU Cluster

引用

27th IEEE international conference on parallel and distributed Systems, ICPADS 2021

作者： Zhang, Yi Goswami, Dhrubajyoti Concordia University Department of Computer Science and Software Engineering Montreal Canada

ISBN: (纸本)9781665408783

Research on Graph Convolutional Networks (GCNs) has increasingly gained popularity in recent years due to the powerful representational capacity of graphs. A common assumption in traditional synchronous parallel training of GCNs using multiple GPUs is that load is perfectly balanced. However, this assumption may not hold in a real-world scenario where there can be imbalances in workloads among GPUs for various reasons. In a synchronous parallel implementation, a straggler in the system can limit the overall speed up of parallel training. To address these performance issues, this research investigates approaches for asynchronous decentralized parallel training of GCNs on a GPU cluster. The techniques investigated are based on graph clustering and the Gossip protocol. The research specifically adapts the approach of Cluster GCN, which uses graph partitioning for SGD based training, and combines with a gossip algorithm specifically designed for a GPU cluster to periodically exchange gradients among randomly chosen partners (GPUs). In addition, it incorporates a work pool mechanism for load balancing among GPUs. The gossip algorithm is proven to be deadlock free. The implementation is performed on a deep learning cluster with 8 Tesla V100 GPUs per compute node, and PyTorch and DGL as the software platforms. Experiments are conducted on different benchmark datasets. The results demonstrate superior performance with similar accuracy scores, as compared to traditional synchronous training which uses 'all reduce' to synchronously accumulate parallel training results. © 2021 IEEE.

关键词： Graphics processing unit

来源：评论

学校读者我要写书评

暂无评论

A p2p environment to validate ensemble-based approaches in the cybersecurity domain 28

A p2p environment to validate ensemble-based approaches in t...

引用

28th Euromicro international conference on parallel, distributed and Network-Based processing (PDP)

作者： Folino, Francesco Folino, Gianluigi Pontieri, Luigi CNR Inst High Performance Comp & Networking ICAR Arcavacata Di Rende CS Italy

ISBN: (数字)9781728165820

ISBN: (纸本)9781728165820

The main data-driven techniques for detecting cybersecurity attacks are based on the analysis of network traffic data and/or of application/system logs (stored in a host or in some other kind of device). A wide range of machine-learning techniques (and possible alternative configurations of them) have been proposed in the literature so far, for this purpose, but none of them has been proven to consistently overcome the others across different datasets. In order to ensure better accuracy and stability, the ensemble paradigm can be exploited as an effective solution for combining such techniques. However, as attack detection problems are hard to cope with and, usually, entail the analysis of large and fast streams of data, different types of ensemble (and of base algorithms composing the ensemble) should have experimented, exploiting distributed architecture to suitably reduce the high-execution times necessary to run them. In order to handle all these issues, a p2p environment to validate ensemble-based approaches in the cybersecurity domain is proposed in this paper. Two case studies are analyzed by using this framework, which concern the detection of intrusions in network-traffic data and of deviant process instances. Preliminary scalability results demonstrate that the framework is a viable solution for these challenging kind of problems.

关键词： Ensemble-based Systems p2p frameworks cybersecurity applications

来源：评论

学校读者我要写书评

暂无评论

Advanced techniques in Data Ingestion and Pipelining for Scalable Big Data Platforms: A Comprehensive Review

Advanced Techniques in Data Ingestion and Pipelining for Sca...

引用

international conference on ICT in Business Industry & Government (ICTBIG)

作者： Sachin Murarka Anshuj Jain Laxmi Singh Department of Electronics & Communication Engineering Rabindranath Tagore University Bhopal India Department of Electronics & Communication Engineering Scope Global University Bhopal India

ISBN: (数字)9798331518981

ISBN: (纸本)9798331518998

In the era of big data, effective data ingestion and pipelining are critical for organizations to harness the power of vast and diverse data sources, such as IoT devices, social media, and transactional systems. Data ingestion refers to the process of systematically collecting and importing data for processing, while data pipelining ensures the continuous flow of this data through various transformation stages, preparing it for analysis. As data volumes grow, platforms like Hadoop, with its HDFS (Hadoop distributed File System) and MapReduce, have become essential in managing large-scale datasets by providing distributed storage and parallel processing capabilities. Additionally, tools such as Apache Kafka and Apache Flume facilitate both batch and real-time data ingestion, ensuring flexibility in handling different types of data streams. However, challenges remain, especially with the increasing complexity of datasets and the growing need for real-time processing. Emerging technologies such as cloud-native ingestion frameworks, edge computing, and AI-enhanced pipelines offer solutions to these challenges by improving scalability, performance, and cost efficiency. Machine learning is also playing an evolving role in optimizing pipeline workflows, enabling more intelligent and adaptive data processing. This paper reviews key technologies for data ingestion and pipelining, discusses their limitations, and explores cutting-edge trends aimed at enhancing the efficiency of big data platforms. These advancements promise to reshape how organizations handle big data, offering improved agility and operational effectiveness. In addition to established technologies, the rise of cloud-native architectures and AI-driven solutions is revolutionizing data ingestion and pipelining. Cloud platforms offer scalable, on-demand resources, while AI and machine learning enhance automation, anomaly detection, and predictive analytics within pipelines.

关键词： Technological innovation Data ingestion Reviews Scalability Machine learning Big Data Data processing Real-time systems Streams Pipeline processing

来源：评论

学校读者我要写书评

暂无评论

Product collaborative filtering based recommendation systems for large-scale E-commerce

引用

international Journal of Information Management Data Insights 2025年第1期5卷

作者： Trinh, Trang Nguyen, Van-Ho Nguyen, Nghia Nguyen, Duy-Nghia University of Economics and Law Ho Chi Minh City Viet Nam Vietnam National University Ho Chi Minh City Viet Nam

The rapid growth in e-commerce and the increasing diversity of customer preferences necessitates the development of an effective recommender system for a business offering a wide range of products. This paper introduces a product-based collaborative filtering approach utilizing Apache Spark, a powerful parallel processing framework to address the scalability issues of recommender systems in the cloud computing environment. Using Spark's distributed computing ability, our model attains a surprising 7.6 times speedup on the training time compared to traditional single-machine methods while preserving accuracy with a Root Mean Square Error (RMSE) 0.9. These results demonstrate the effectiveness of parallel and distributed techniques in developing efficient and accurate recommender systems for large-scale e-commerce applications. Future work will focus on applying multi-model to enhance the accuracy of prediction and configuration to optimize the cost of cluster operations. © 2025 The Author(s)

关键词： Apache spark Collaborative filtering E-commerce Large-scale parallel and distributed computing Recommendation systems

来源：评论

学校读者我要写书评

暂无评论

A Novel Approach to Braun Multiplier Design Utilizing High-Speed parallel Prefix Adder for Low Power applications

A Novel Approach to Braun Multiplier Design Utilizing High-S...

引用

international conference on Computation System and Information Technology for Sustainable Solutions (CSITSS)

作者： Kiran V E M Savi Teja Department of ECE RV College of Engineering Bengaluru India

This project develops an innovative Braun Multiplier design to address power consumption and chip area challenges. Integrating a high-speed parallel prefix adder enhances computational speed by leveraging parallel processing capabilities. The synergy between the Braun Multiplier and the adder yields a solution optimizing performance, power efficiency, and chip area usage. Meticulous design and engineering yield notable improvements in power efficiency and chip area utilization. The project drives advancements in digital circuit design and optimization methodologies, achieving a 16% area reduction and 18.2% lower power consumption via the FS-GDI technique, while achieving full-swing output voltage, distinguishing it from GDI and M-GDI techniques.

关键词：

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：