检索结果-内蒙古大学图书馆

Energy-time modelling of distributed multi-population genetic algorithms with dynamic workload in HPC clusters

FUTURE GENERATION COMPUTER SYSTEMS-THE INTERNATIONAL JOURNAL OF ESCIENCE 2025年 167卷

作者： Escobar, Juan Jose Sanchez-Cuevas, Pablo Prieto, Beatriz Kiziltepe, Rukiye Savran Diaz-del-Rio, Fernando Kimovski, Dragi Univ Granada Dept Software Engn CITIC Granada Spain Univ Seville Dept Comp Architecture & Technol Seville Spain Univ Granada Dept Comp Engn Automat & Robot CITIC Granada Spain Karadeniz Tech Univ Dept Software Engn Trabzon Turkiye Ankara Univ Dept Software Engn Ankara Turkiye Univ Klagenfurt Inst Informat Technol Klagenfurt Austria

Time and energy efficiency is a highly relevant objective in high-performance computing systems, with high costs for executing the tasks. Among these tasks, evolutionary algorithms are of consideration due to their inherent parallel scalability and usually costly fitness evaluation functions. In this respect, several scheduling strategies for workload balancing in heterogeneous systems have been proposed in the literature, with runtime and energy consumption reduction as their goals. Our hypothesis is that a dynamic workload distribution can befitted with greater precision using metaheuristics, such as genetic algorithms, instead of linear regression. Therefore, this paper proposes anew mathematical model to predict the energy-time behaviour of applications based on multi-population genetic algorithms, which dynamically distributes the evaluation of individuals among the CPU-GPU devices of heterogeneous clusters. An accurate predictor would save time and energy by selecting the best resource set before running such applications. The estimation of the workload distributed to each device has been carried out by simulation, while the model parameters have been fitted in a two-phase run using another genetic algorithm and the experimental energy-time values of the target application as input. When the new model is analysed and compared with another based on linear regression, the one proposed in this work significantly improves the baseline approach, showing normalised prediction errors of 0.081 for runtime and 0.091 for energy consumption, compared to 0.213 and 0.256 shown in the baseline approach.

关键词： Energy-time modelling Heterogeneous clusters distributed computing Parameter optimisation Task scheduling Genetic algorithms

来源：评论

学校读者我要写书评

暂无评论

A distributed Information Granulation Method for Time Series Clustering

引用

CONCURRENCY AND COMPUTATION-PRACTICE & EXPERIENCE 2025年第4-5期37卷

作者： Mu, Yashuang Liu, Tian Zhang, Wenqiang Guo, Hongyue Wang, Lidong Liu, Xiaodong Henan Univ Technol Sch Artificial Intelligence & Big Data Zhengzhou Peoples R China Henan Univ Technol Coll Informat Sci & Engn Zhengzhou Peoples R China Dalian Maritime Univ Sch Maritime Econ & Management Dalian Peoples R China Dalian Maritime Univ Collaborat Innovat Ctr Transport Studies Dalian Peoples R China Dalian Maritime Univ Sch Sci Dalian Peoples R China Dalian Univ Technol Fac Elect Informat & Elect Engn Sch Control Sci & Engn Dalian Peoples R China

Time series clustering is an important research problem in machine learning and data mining. With the rapid increase in the amount of time series data, many traditional clustering algorithms cannot directly deal with large-scale time series due to some limitations in the memory capacity and the execution time. In this study, we suggest a distributed information granulation method for large-scale time clustering problem. First, a distributed time series partitioning method is designed to randomly divide the original time series dataset into some data blocks. Then, the distributed time series granulation method is developed in the map-reduce framework by the principle of reasonable granularity, where each time series can be described by some representative data points to show the trend state information. Finally, we introduce the large-scale time series clustering method in terms of the fuzzy C-means clustering algorithm. The experimental studies demonstrate the feasibility and the effectiveness on several UCR publicly benchmark time series datasets. Compared with the classical clustering methods, the proposed method can achieve a 4.86-9.65% improvement in average clustering accuracy. Meanwhile, the proposed method exhibits more advantages in both unequal length time series clustering and execution time.

关键词： clustering distributed computing information granulation time series

来源：评论

学校读者我要写书评

暂无评论

IDAD: An improved tensor train based distributed DDoS attack detection framework and its application in complex networks

引用

FUTURE GENERATION COMPUTER SYSTEMS-THE INTERNATIONAL JOURNAL OF ESCIENCE 2025年 162卷

作者： Fan, Qiyuan Li, Xue Wang, Puming Jin, Xin Yao, Shaowen Miao, Shengfa An, Min Zhao, Yuqing Yunnan Univ Sch Software Kunming 650091 Peoples R China Henan Univ Sci & Technol Sch Elect Informat Engn Xinxiang 453003 Peoples R China Hubei Univ Automot Technol Shiyan 442000 Peoples R China

With the vigorous development of Internet technology, the scale of systems in the network has increased sharply, which provides a great opportunity for potential attacks, especially the distributed Denial of Service (DDoS) attack. In this case, detecting DDoS attacks is critical to system security. However, current detection methods exhibit limitations, leading to compromises in accuracy and efficiency. To cope with it, three key strategies are implemented in this paper: (i) Using tensors to model large-scale and heterogeneous data in complex networks;(ii) Proposing a denoising algorithm based on the improved and distributed tensor train (IDTT) decomposition, which optimizes the tensor train(TT) decomposition in terms of parallel computation and low-rank estimation;(iii) Combining (i), (ii) and Light Gradient Boosting Machine (LightGBM) classification model, an efficient DDoS attack detection framework is proposed. Datasets CIC-DDoS2019 and NSL-KDD are used to evaluate the framework, and results demonstrate that accuracy can reach 99.19% while having the characteristics of low storage consumption and well speedup ratio.

关键词： Tensor train decomposition DDoS attack detection distributed computing System storage efficiency Machine learning

来源：评论

学校读者我要写书评

暂无评论

Multi-attribute-based self-stabilizing algorithm for leader election in distributed systems

引用

JOURNAL OF SUPERcomputing 2025年第4期81卷 1-35页

作者： Biswas, Amit Singh, Manisha Baranwal, Gaurav Tripathi, Anil Kumar Aknine, Samir Motilal Nehru Natl Inst Technol Allahabad Dept Comp Sci & Engn Prayagraj India Maulana Azad Natl Inst Technol Dept Comp Sci & Engn Bhopal India Banaras Hindu Univ Dept Comp Sci Varanasi India Indian Inst Technol BHU Dept Comp Sci & Engn Varanasi India Univ Claude Bernard Lyon 1 LIRIS lab Lyon France

Cloud computing, edge computing, replicated databases, and various modern technologies adopt distributed computing concepts for a reliable, high-performance, large-scale computational platform. In distributed computing, leader election is a fundamental problem because the elected leader helps coordinate and utilize the resources efficiently. Several state-of-the-art works mentioned that a good quality leader is essential as it improves system performance, simplifies management procedures, reduces coordinational complexity, and makes the system more fault-tolerant. However, designing a self-stabilizing leader election algorithm with weak assumptions in a failure-prone distributed environment is challenging. This paper proposes a multi-attribute-based, self-stabilizing, network partition-tolerant leader election method for failure-prone distributed systems. Here, based on the system requirements, the pertinent attributes of a good quality leader are identified and assigned weights according to their importance. Then, the identified attributes and their weight are used to elect a suitable node as the leader. We show that the algorithm is self-stabilizing and can tolerate multiple nodes and link failures. Further, we analyze the proposed algorithm's time, communication, and bit complexities. We consider a distributed database system scenario to simulate the proposed method and compare it with existing approaches to evaluate and validate the proposed method's performance and the elected leader's quality.

关键词： Leader election distributed computing distributed systems Multi-attribute decision-making Group decision-making

来源：评论

学校读者我要写书评

暂无评论

Hybrid in-network computing and distributed learning for large-scale data processing

引用

COMPUTER NETWORKS 2023年 226卷

作者： Jeon, So-Eun Lee, Sun-Jin Lee, Il-Gu Sungshin Womens Univ Dept Future Convergence Technol Engn 34 Da GilBomun Ro Seoul 02844 South Korea

Recently, the value of big data generated, processed, and shared by mobile devices and intelligent Internet of Things devices has significantly increased. The speed of data generation is accelerating, and the amount of data that needs to be collected and learned in a distributed database is increasing. However, the conventional data processing methods, centralized and distributed computing, are reaching their limits owing to the trade-off between data processing speed and learning accuracy. To solve this problem, in this study, a hybrid in -network computing (HINC) method that can process data quickly and accurately is proposed. The efficiency of the proposed HINC method was verified by comparing and analyzing its data loss, throughput, and accuracy to those of the conventional method. The experimental results show that in an environment with many nodes in the network, HINC reduces the data loss, which means data that could not be processed because it exceeded the available throughput of the node by 20.94% compared to the conventional method. Furthermore, in an envi-ronment with full workload, HINC reduced data loss by an average of 55.6% compared with the conventional centralized computation method. It also improved throughput by an average of 18.91% and model evaluation accuracy by 15.7% compared with the conventional fully distributed computing model.

关键词： Centralized computing distributed computing In-network computing Data processing Large-scale data

来源：评论

学校读者我要写书评

暂无评论

Object Proxy Patterns for Accelerating distributed Applications

引用

IEEE TRANSACTIONS ON PARALLEL AND distributed SYSTEMS 2025年第2期36卷 253-265页

作者： Pauloski, J. Gregory Hayot-Sasson, Valerie Ward, Logan Brace, Alexander Bauer, Andre Chard, Kyle Foster, Ian Univ Chicago Dept Comp Sci Chicago IL 60637 USA Argonne Natl Lab Data Sci & Learning Div Lemont IL 60439 USA IIT Dept Comp Sci Chicago IL 60616 USA

Workflow and serverless frameworks have empowered new approaches to distributed application design by abstracting compute resources. However, their typically limited or one-size-fits-all support for advanced data flow patterns leaves optimization to the application programmer-optimization that becomes more difficult as data become larger. The transparent object proxy, which provides wide-area references that can resolve to data regardless of location, has been demonstrated as an effective low-level building block in such situations. Here we propose three high-level proxy-based programming patterns-distributed futures, streaming, and ownership-that make the power of the proxy pattern usable for more complex and dynamic distributed program structures. We motivate these patterns via careful review of application requirements and describe implementations of each pattern. We evaluate our implementations through a suite of benchmarks and by applying them in three meaningful scientific applications, in which we demonstrate substantial improvements in runtime, throughput, and memory usage.

关键词： distributed databases Genomics Computational modeling Production facilities Engines Connectors Programming Memory management Load modeling Complexity theory distributed computing futures memory management open-source software streaming

来源：评论

学校读者我要写书评

暂无评论

DRLFluent: A distributed co-simulation framework coupling deep reinforcement learning with Ansys-Fluent on high-performance computing systems

引用

JOURNAL OF COMPUTATIONAL SCIENCE 2023年 74卷

作者： Mao, Yiqian Zhong, Shan Yin, Hujun Univ Manchester Dept Mech Aerosp & Civil Engn Manchester M13 9PL England Univ Manchester Dept Elect & Elect Engn Manchester M13 9PL England

For active flow control (AFC), several frameworks have been developed to enable dynamic interactions between deep reinforcement learning (DRL) agents and computational fluids dynamics (CFD) environments. However, the highly coupled modules in the existing frameworks require massive development efforts to be applied to different flow scenarios. Moreover, these frameworks do not support applications to run flexibly on multiple nodes in high-performance computing (HPC) systems. Herein, we propose a new framework named as DRLFluent coupling an open-source DRL package with a well-developed general CFD solver, Ansys-Fluent. The distributed HPC deployment of DRLFluent is achieved by a broker architecture which enables the DRL client and CFD server written in different languages to communicate across different nodes via a pair of standardized interfaces. We have evaluated DRLFluent along with two other representative frameworks using a benchmark case in which a laminar flow around a circular cylinder (Re=100) is controlled by two synthetic jets. The strategy discovered in DRLFluent is capable of neutralizing the small asymmetry of the shedding vortices, yielding a higher drag reduction than the other two frameworks. When the number of parallel environments exceeds 20, DRLFluent exhibits the largest speedup among the frameworks evaluated. Capable of distributing workload across multiple nodes, DRLFluent is suitable for implementing DRL for AFC of complex flow scenarios which requires hundreds of cores or more. Benefitting from a loose coupling between DRL and CFD, the standardized interfaces and the user-friendly CFD solver, DRLFluent is readily applicable to a wide range of flow scenarios without laborious low-level programming efforts.

关键词： Computational fluid dynamics Reinforcement learning High -performance computing distributed computing Flow control

来源：评论

学校读者我要写书评

暂无评论

A distributed approach for persistent homology computation on a large scale

引用

JOURNAL OF SUPERcomputing 2024年第17期80卷 25510-25532页

作者： Ceccaroni, Riccardo Di Rocco, Lorenzo Ferraro Petrillo, Umberto Brutti, Pierpaolo Univ Roma La Sapienza Dept Stat Sci Ple Aldo Moro 5 I-00185 Rome Italy

Persistent homology (PH) is a powerful mathematical method to automatically extract relevant insights from images, such as those obtained by high-resolution imaging devices like electron microscopes or new-generation telescopes. However, the application of this method comes at a very high computational cost that is bound to explode more because new imaging devices generate an ever-growing amount of data. In this paper, we present PixHomology, a novel algorithm for efficiently computing zero-dimensional PH on 2D images, optimizing memory and processing time. By leveraging the Apache Spark framework, we also present a distributed version of our algorithm with several optimized variants, able to concurrently process large batches of astronomical images. Finally, we present the results of an experimental analysis showing that our algorithm and its distributed version are efficient in terms of required memory, execution time, and scalability, consistently outperforming existing state-of-the-art PH computation tools when used to process large datasets.

关键词： Persistent homology distributed computing Apache spark Large-scale image analysis

来源：评论

学校读者我要写书评

暂无评论

Data sharing mode of dispatching automation system based on distributed machine learning

引用

INTERNATIONAL JOURNAL OF NETWORK MANAGEMENT 2025年第1期35卷

作者： He, Xiaoli Luo, Mi Hu, Yurui Xiong, Feng Chongqing Chem Ind Vocat Coll Sch Big Data & Automat Engn Chongqing Peoples R China Chongqing Chem Ind Vocat Coll Sch Big Data & Automat Engn Chongqing 401228 Peoples R China

The difficulties of sending massive amounts of data between several data centres are examined in this work, with particular attention paid to how poorly current scheduling algorithms handle point-to-multipoint transfers and transmission time limits. In this research, a new method called multicast source-based tree (MSBT) is proposed for effectively handling point-to-multipoint transmissions in a certain amount of time. By allowing receivers to simultaneously receive data from several source points, MSBT introduces the idea of 'source selection' for the creation of multicast tree structure-based algorithms. Large data blocks are distributed as efficiently as possible using this method, which also guarantees effective transmission from a single-source point to several recipient locations. Furthermore covered in the article is how PV producers and sellers' capacity allocation is affected by the discount rate. These results offer insightful information on how decisions are made in related sectors. The development of new energy big data platforms underscores their significance;leaders in the industry, like United Power, Vision Energy and Goldwind, serve as prime examples. In this paper, a new method called multicast source-based tree (MSBT) is proposed for effectively handling point-to-multipoint transmissions in a certain amount of time. By allowing receivers to simultaneously receive data from several source points, MSBT introduces the idea of 'source selection' for the creation of multicast tree structure-based algorithms. Large data blocks are distributed as efficiently as possible using this method, which also guarantees effective transmission from a single-source point to several recipient locations. Furthermore covered in the article is how PV producers and sellers' capacity allocation is affected by the discount rate. These results offer insightful information on how decisions are made in related sectors. image

关键词： AI platform big data data locality dispatching system distributed computing seize

来源：评论

学校读者我要写书评

暂无评论

A Safe and Efficient Global Path-Planning Method Considering Multiple Environmental Factors of the Moon Using a distributed Computation Strategy

引用

REMOTE SENSING 2025年第5期17卷 924-924页

作者： Zhou, Ruyan Liu, Yuchuan Hong, Zhonghua Pan, Haiyan Zhang, Yun Han, Yanling Tao, Jiang Shanghai Ocean Univ Coll Informat Technol Shanghai 201306 Peoples R China

Lunar-rover path planning is a key topic in lunar exploration research, with safety and computational efficiency critical for achieving long-distance planning. This paper proposes a distributed path-planning method that considers multiple lunar environmental factors, addressing the issues of inadequate safety considerations and low computational efficiency in current research. First, a set of safety evaluation rules is constructed by considering factors such as terrain slope, roughness, illumination, and rock abundance. Second, a distributed path-planning strategy based on a safety-map tile pyramid (DPPS-STP) is proposed, using a weighted A* algorithm with hash table-based open and closed lists (OC-WHT-A*) on a Spark cluster for efficient and safer path planning. Additionally, high-resolution digital orthophoto maps (DOM) are utilized for small crater detection, enabling more refined path planning built upon the overall mission-planning result. The method was validated in four lunar regions with distinct characteristics. The results show that DPPS-STP, which considers multiple environmental factors, effectively reduces the number of hazardous nodes and avoids crater obstacles. For long-distance tasks, it achieves an average speedup of up to 11.5 times compared to the single-machine OC-WHT-A*, significantly improving computational efficiency.

关键词： path planning lunar A* algorithm distributed computing crater detection

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：