检索结果-内蒙古大学图书馆

Accelerating Distributed Repartition Joins on Skewed Datasets via Patch-Based Shuffling

IEEE ACCESS 2025年 13卷 41068-41096页

作者： Kassela, Evdokia Konstantinou, Ioannis Koziris, Nectarios Natl & Tech Univ Athens Sch Elect & Comp Engn Athens 11527 Greece Univ Thessaly Dept Informat & Telecommun Lamia 35100 Greece

In distributed workloads involving joins and aggregations, skewed attribute values often cause load balancing issues, leading to stragglers and increased execution times. Existing solutions often rely on cost-based models, require extensive parameter tuning, or necessitate modifications to distributed execution engines, limiting their usability and generality. To address this challenge, we present a novel patch-based repartitioning algorithm that eliminates load imbalances while minimizing network overhead. Our approach extends the subset-replicate technique by leveraging data distribution and location statistics to optimize data locality and reduce unnecessary data movement. Unlike traditional hash-based methods, our technique is skew-insensitive, requires no parameter tuning, and integrates seamlessly into existing distributed execution engines as a drop-in replacement for the shuffle mechanism. The proposed method operates in three distinct stages: (1) statistics collection and load threshold computation, (2) patch-based subgroup assignment to ensure optimal load balancing with minimal replication, and (3) informed data shuffling and join execution. This structured process ensures even workload distribution across workers while reducing network I/O. Theoretical analysis proves the scalability, skew robustness, and load-balancing guarantees of our approach, establishing bounds on maximum worker load and network data movement. Experimental evaluations demonstrate that our method achieves perfectly balanced workloads and reduces execution time by up to 81% compared to conventional hash-based joins under moderate to high skew, while introducing negligible overhead at low skew levels. These improvements are attributed to reduced data movement, optimized worker utilization, and the algorithm's robust theoretical foundation. Our research provides a versatile and practical solution for skewed data processing, significantly advancing the efficiency of distributed data man

关键词： Engines partitioning algorithms Load management Distributed databases Costs Telecommunication traffic Optimization Logic Load modeling Histograms Load balancing repartition join shuffling subset-replicate

来源：评论

学校读者我要写书评

暂无评论

GBCT: Efficient and Adaptive Clustering via Granular-Ball Computing for Complex Data

引用

IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS 2025年 PP卷 PP页

作者： Xia, Shuyin Shi, Bolun Wang, Yifan Xie, Jiang Wang, Guoyin Gao, Xinbo Chongqing Univ Posts & Telecommun Chongqing Key Lab Computat Intelligence Key Lab Cyberspace Big Data Intelligent Secur Minist Educ Chongqing 400065 Peoples R China Chongqing Univ Posts & Telecommun Key Lab Big Data Intelligent Comp Chongqing 400065 Peoples R China Chongqing Univ Posts & Telecommun Dept Comp Sci Chongqing 400065 Peoples R China

Traditional clustering algorithms often focus on the most fine-grained information and achieve clustering by calculating the distance between each pair of data points or implementing other calculations based on points. This way is not inconsistent with the cognitive mechanism of "global precedence" in the human brain, resulting in those methodsbad performance in efficiency, generalization ability, and robustness. To address this problem, we propose a new clustering algorithm called granular-ball clustering via granular-ball computing. First, clustering algorithm based on granular-ball (GBCT) generates a smaller number of granular-balls to represent the original data and forms clusters according to the relationship between granular-balls, instead of the traditional point relationship. At the same time, its coarse-grained characteristics are not susceptible to noise, and the algorithm is efficient and robust;besides, as granular-balls can fit various complex data, GBCT performs much better in nonspherical datasets than other traditional clustering methods. The completely new coarse granularity representation method of GBCT and cluster formation mode can also be used to improve other traditional methods.

关键词： Clustering algorithms partitioning algorithms Computational modeling Robustness Noise Approximation algorithms Mathematical models Clustering methods Telecommunications Shape Clustering granular computing granular-ball multigranularity

来源：评论

学校读者我要写书评

暂无评论

Group Synchronization of Heterogeneous Multiagent System With Hierarchical Structure and Beyond Pairwise Interactions

引用

IEEE TRANSACTIONS ON SYSTEMS MAN CYBERNETICS-SYSTEMS 2025年第3期55卷 1578-1590页

作者： Chen, Lin Xia, Yuanqing Zhang, Yuan Beijing Inst Technol Sch Automat Beijing 100081 Peoples R China Zhongyuan Univ Technol Zhengzhou 450007 Henan Peoples R China

Systems of different types of agents coupled with high-order interactions can emerge complex synchronization patterns. This article presents an analysis framework to study the group synchronization of heterogeneous multiagent systems with hierarchical structure and directed interactions of arbitrary order. First, directed hypergraph is introduced to describe the arbitrary-order interactions among the agents. Then, to explore the possible group synchronization patterns, the agent partition methods commonly used in dyadic graphs are extended to the directed hypergraph. Meanwhile, to verify the admissibility of the group synchronization patterns, two equitable partition types of agent partition are defined from the perspective of hyperedge partition. Next, using the simultaneous block diagonalization method, the stability problem of the group synchronous solution is decoupled into several independent subblocks. These subblocks are associated with the perturbations along and transverse to the synchronous manifold, with transverse subblocks related to loss of synchronization. Therefore, the stability of the group synchronous solution can be investigated by evaluating the stability of each transverse subblock via the master stability function method, thus significantly reducing the stability calculation complexity. Finally, the effectiveness of the analysis framework is illustrated via the simulation.

关键词： Synchronization Multi-agent systems Stability criteria Circuit stability partitioning algorithms Neurons Nonhomogeneous media Tensors Social networking (online) Mathematical models Directed hypergraph group synchronization heterogeneous multiagent systems high-order interactions master stability function (MSF)

来源：评论

学校读者我要写书评

暂无评论

Fuzzy Min-Cut With Soft Balancing Effects

引用

IEEE TRANSACTIONS ON FUZZY SYSTEMS 2025年第2期33卷 767-778页

作者： Chen, Huimin Zhang, Runxin Wang, Rong Nie, Feiping Northwestern Polytech Univ Sch Comp Sci Sch Artificial Intelligence Opt & Elect iOPEN Xian 710072 Peoples R China Northwestern Polytech Univ Key Lab Intelligent Interact & Applicat Minist Ind & Informat Technol Xian 710072 Peoples R China Northwestern Polytech Univ Sch Artificial Intelligence OPt & Elect iOPEN Xian 710072 Peoples R China

The clustering algorithm has always been a hot spot in machine learning, which has made great progress and been widely used in different scenarios. Due to the characteristics and requirements of some application scenarios, the branch of the balanced clustering algorithm has been developed. The ideal of these algorithms is to obtain clusters containing approximately the same number of samples. However, when there are data points distributed at the boundary of different clusters, resulting in different probabilities of their belonging, hard-partitioned balanced clustering may not be able to handle these boundary data well, thus limiting their performance. Motivated by this, we propose a Fuzzy Min-Cut with Soft Balancing Effects (FCBE) method in this article. Specifically, the FCBE model utilizes fuzzy constraints to simultaneously enhance the ability of the balanced algorithm to capture boundary data members and the advantage of directly obtaining the partitioning results of graph-cut problem without postprocessing. In addition, a sparse regularization is introduced to avoid trivial solutions and maintain the separability of the relationship matrix. Furthermore, the proposed FCBE method can be viewed as a flexibly adjustable generalization pattern that not only has clear interpretability but also can become special cases with clear physical meanings under different parameter values. The feasibility of FCBE has been verified on real datasets.

关键词： Clustering algorithms partitioning algorithms Fuzzy systems Vectors Sparse matrices Switches Data models Clustering methods Wireless sensor networks Uncertainty Fuzzy constraints soft-balanced sparse regularization spectral clustering (SC)

来源：评论

学校读者我要写书评

暂无评论

An Energy Effective RIS-Assisted Multi-UAV Coverage Scheme for Fairness-Aware Ground Terminals

IEEE TRANSACTIONS ON GREEN COMMUNICATIONS AND NETWORKING

引用

IEEE TRANSACTIONS ON GREEN COMMUNICATIONS AND NETWORKING 2025年第1期9卷 164-176页

作者： Lin, Na Wu, Tianxiong Zhao, Liang Hawbani, Ammar Wan, Shaohua Guizani, Mohsen Shenyang Aerosp Univ Sch Comp Sci Shenyang 110136 Peoples R China Univ Elect Sci & Technol China Shenzhen Inst Adv Study Shenzhen 518110 Peoples R China Mohamed Bin Zayed Univ Artificial Intelligence Machine Learning Dept Abu Dhabi U Arab Emirates

Unmanned aerial vehicle (UAV)-assisted communications are critical in regional wireless networks. Using reconfigurable intelligent surfaces (RISs) can significantly improve UAVs' throughput and energy efficiency. Due to limited communications resources, the data transfer rate of ground terminals (GTs) could be slower, and the throughput may be low. Using RIS-assisted UAVs can effectively address these limitations. This paper focuses on optimizing the three-dimensional (3D) trajectory of the UAV and the scheduling order of the GTs and designing the phase shift of the RIS to maximize energy efficiency while meeting the limited energy and fair service constraints in the case of fair service GTs. To address the non-convexity of this problem, we propose a triple deep q-network (TDQN) algorithm, which better avoids the overestimation problem during the optimization process. We propose an improved k-density-based spatial clustering of applications with noise (K-DBSCAN) clustering algorithm, which is characterized by the ability to output the initial movement range of the UAV and prune the deep reinforcement learning (DRL) state space by the initial movement range to speed up DRL training based on the completion of the partitioning deployment work. A fair screening mechanism is proposed to satisfy the fairness constraint. The results show that the TDQN algorithm is 2.9% more energy efficient than the baseline. The K-DBSCAN algorithm speeds up the training of the TDQN algorithm by 59.4%. The fair screening mechanism reduces the throughput variance from an average of 114099.9 to an average of 46.9.

关键词： Autonomous aerial vehicles Energy efficiency Clustering algorithms Optimization Throughput partitioning algorithms Wireless communication Reconfigurable intelligent surface (RIS) unmanned aerial vehicle (UAV) deep reinforcement learning (DRL) triple deep q network (TDQN)

来源：评论

学校读者我要写书评

暂无评论

Towards Communication-Efficient Out-of-Core Graph Processing on the GPU

引用

IEEE TRANSACTIONS ON PARALLEL AND DISTRIBUTED SYSTEMS 2025年第5期36卷 961-976页

作者： Wang, Qiange Ai, Xin Yan, Yongze Gong, Shufeng Zhang, Yanfeng Chen, Jing Yu, Ge Northeastern Univ Sch Comp Sci & Engn Shenyang 110819 Peoples R China

The key performance bottleneck of large-scale graph processing on memory-limited GPUs is the host-GPU graph data transfer. Existing GPU-accelerated graph processing frameworks address this issue by managing the active subgraph transfer at runtime. Some frameworks adopt explicit transfer management approaches based on explicit memory copy with filter or compaction. In contrast, others adopt implicit transfer management approaches based on on-demand accesses with the zero-copy mechanism or unified virtual memory. Having made intensive analysis, we find that as the active vertices evolve, the performance of the two approaches varies in different workloads. Due to heavy redundant data transfers, high CPU compaction overhead, or low bandwidth utilization, adopting a single approach often results in suboptimal performance. Moreover, these methods lack effective cache management methods to address the irregular and sparse memory access pattern of graph processing. In this work, we propose a hybrid transfer management approach that takes the merits of both two transfer approaches at runtime. Moreover, we present an efficient vertex-centric graph caching framework that minimizes CPU-GPU communication by caching frequently accessed graph data at runtime. Based on these techniques, we present HytGraph, a GPU-accelerated graph processing framework, which is empowered by a set of effective task-scheduling optimizations to improve performance. Experiments on real-world and synthetic graphs show that HytGraph achieves average speedups of 2.5 x, 5.0 x, and 2.0 x compared to the state-of-the-art GPU-accelerated graph processing systems, Grus, Subway, and EMOGI, respectively.

关键词： Graphics processing units Public transportation Memory management Iterative methods Data transfer Costs Runtime partitioning algorithms Bandwidth Parallel processing GPU graph processing communication reduction transfer management out-of-core processing

来源：评论

学校读者我要写书评

暂无评论

Improved Parallel Algorithm for Finding Minimum Cuts in Stochastic Flow Networks

引用

IEEE LATIN AMERICA TRANSACTIONS 2025年第4期23卷 285-293页

作者： Joshan, Mohammad Saito, Jose Pedrino, Emerson Univ Fed Sao Carlos Sao Carlos SP Brazil

This article solves the convergence problem in the Parallel BK algorithm for large-scale flow networks. We introduce a merging method and a pseudo-Boolean representation-based invariance analysis that optimize the algorithm's performance compared to classical approaches such as Ford-Fulkerson, Edmonds-Karp, Push-Relabel, and Kargers Algorithm. Our approach improves energy efficiency, reduces memory usage, and lowers time complexity by leveraging parallelization techniques. We evaluate the performance of these algorithms using Python simulations on various benchmark graphs, ranging from small to large-scale networks. The results show that our method reduces memory consumption by up to 40% and speeds up execution time by 30%, while maintaining high accuracy in finding minimum cuts. This paper demonstrates the algorithm's potential for applications in image segmentation, wireless sensor networks, and network reliability analysis

关键词： Heuristic algorithms Merging Parallel algorithms Convergence Accuracy partitioning algorithms Optimization Graph theory Computational efficiency Reliability Stochastic processes Parallel Algorithm Connectivity Min-Cuts Stochastic-Flow Networks Network Reliability

来源：评论

学校读者我要写书评

暂无评论

Benchmarking Derivative-Free Global Optimization algorithms Under Limited Dimensions and Large Evaluation Budgets

引用

IEEE TRANSACTIONS ON EVOLUTIONARY COMPUTATION 2025年第1期29卷 187-204页

作者： Stripinis, Linas Kudela, Jakub Paulavicius, Remigijus Vilnius Univ Inst Data Sci & Digital Technol LT-08663 Vilnius Lithuania Brno Univ Technol Inst Automat & Comp Sci Brno 61600 Czech Republic

This article addresses the challenge of selecting the most suitable optimization algorithm by presenting a comprehensive computational comparison between stochastic and deterministic methods. The complexity of algorithm selection arises from the absence of a universal algorithm and the abundance of available options. Manual selection without comprehensive studies can lead to suboptimal or incorrect results. In order to address this issue, we carefully selected 25 promising and representative state-of-the-art algorithms from both aforementioned classes. The evaluation with up to the 20 dimensions and large evaluation budgets $(10<^>{5}{\times }n)$ was carried out in a significantly expanded and improved version of the DIRECTGOLib v2.0 library, which included ten distinct collections of primarily continuous test functions. The evaluation covered various aspects, such as solution quality, time complexity, and function evaluation usage. The rankings were determined using statistical tests and performance profiles. When it comes to the problems and algorithms examined in this study, EA4eig, EBOwithCMAR, APGSK-IMODE, 1-DTC-GL, OQNLP, and DIRMIN stand out as superior to other derivative-free solvers in terms of solution quality. While deterministic algorithms can locate reasonable solutions with comparatively fewer function evaluations, most stochastic algorithms require more extensive evaluation budgets to deliver comparable results. However, the performance of stochastic algorithms tends to excel in more complex and higher-dimensional problems. These research findings offer valuable insights for practitioners and researchers, enabling them to tackle diverse optimization problems effectively.

关键词： Optimization partitioning algorithms Evolutionary computation Benchmark testing Stochastic processes Linear programming Closed box Derivative-free global optimization deterministic algorithms evolutionary computation (EC) algorithms nature-inspired meta-heuristics numerical benchmarking

来源：评论

学校读者我要写书评

暂无评论

Maximizing Influence Query Over Indoor Trajectories

引用

IEEE TRANSACTIONS ON KNOWLEDGE AND DATA ENGINEERING 2025年第3期37卷 1294-1310页

作者： Chen, Jian Gao, Hong Shi, Yuhong Chen, Junle Yang, Donghua Li, Jianzhong Harbin Inst Technol Dept Comp Sci & Technol Harbin 150001 Peoples R China Zhejiang Normal Univ Coll Math & Comp Sci Jinhua 321004 Peoples R China Chinese Acad Sci Shenzhen Inst Adv Technol Shenzhen 518055 Peoples R China

Maximizing Influence (Max-Inf) query is a fundamental operation in spatial data management. This query returns an optimal site from a candidate set to maximize its influence. Existing work commonly focuses on outdoor spaces. In practice, however, people spend up to 87% of their daily life inside indoor spaces. The outdoor techniques fall short in indoor spaces due to the complicated topology of indoor spaces. In this paper, we formulate two indoor Max-Inf queries: Top-kk Probabilistic Influence Query (TkkPI) and Collective-kk Probabilistic Influence Query (CkkPI) taking probability and mobility factors into consideration. We propose a novel spatial index, IT-tree, which utilizes the properties of indoor venues to facilitate the indoor distance computation, and then applies a trie to further organize the trajectories with similar check-in partitions together, based on their sketch information. This structure is simple but highly effective in pruning the trajectory search space. To process TkPI efficiently, we devise subtree pruning and progressive pruning techniques to delicately filter out unnecessary trajectories based on probability bounds and the monotonicity of influence probability. For CkPI queries, which is a submodular NP-hard problem, three approximation algorithms are provided with different strategies of computing marginal influence value during the search. Through extensive experiments on several real indoor venues, we demonstrate the efficiency and effectiveness of our proposed algorithms.

关键词： Trajectory Approximation algorithms Probabilistic logic Indexing Emergency services Costs Topology Spatial indexes Roads partitioning algorithms Indoor query pruning strategy spatial index

来源：评论

学校读者我要写书评

暂无评论

Optimized Subset Selection for Matrices and its Application in Hyperspectral Band Selection

引用

IEEE JOURNAL OF SELECTED TOPICS IN APPLIED EARTH OBSERVATIONS AND REMOTE SENSING 2025年 18卷 6464-6479页

作者： Xiao, Songyi Zhu, Liangliang Zhu, Xinwen Geng, Xiurui Chinese Acad Sci Aerosp Informat Res Inst Beijing 100094 Peoples R China Chinese Acad Sci Key Lab Technol Geospatial Informat Proc & Applica Beijing 100190 Peoples R China Univ Chinese Acad Sci Sch Elect Elect & Commun Engn Beijing 100049 Peoples R China Chinese Acad Sci Key Lab Target Cognit & Applicat Technol Beijing 100190 Peoples R China Hefei Univ Technol Sch Comp Sci & Informat Engn Hefei 230009 Peoples R China

When spectral bands are considered as scatter points in the image space, hyperspectral band selection can be transformed into the column subset selection problem for matrices. We consider the problem of finding the submatrix of maximum volume of a matrix X is an element of R-NxL (N >> L) within the column space perspective. To solve this problem, we present a novel determinant formula that utilizes the scalar product of arbitrary-order antisymmetric tensors in wedge product form, providing a fresh perspective for interpreting the volume gradient. Based on the variant of this formula, we can mitigate the local extremum problem inherent in existing subset selection algorithms to a certain extent. Subsequently, we introduce exterior algebra theory into the remote sensing community and develop a hyperspectral band selection algorithm via higher order joint volume gradient. To address bias in the volume criterion and reduce algorithmic complexity, we integrate a general subspace partitioning framework that accounts for spectral information. In addition, we propose a method for automatically determining the optimal number of bands. Experiments conducted on both publicly available and Gaofen-5 satellite hyperspectral datasets demonstrate the superiority and stability of our proposed algorithms in terms of classification accuracy and computational efficiency.

关键词： Hyperspectral imaging partitioning algorithms Classification algorithms Vectors Optimization Greedy algorithms Approximation algorithms Accuracy Tensors Symmetric matrices Band selection (BS) column subset selection problem (CSSP) Gaofen-5 (GF-5) local extremum problem wedge product

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：