检索结果-内蒙古大学图书馆

Dynamic Replication Policy on HDFS Based on Machine Learning Clustering

IEEE ACCESS 2023年 11卷 18551-18559页

作者： Ahmed, Motaz A. Khafagy, Mohamed H. Shaheen, Masoud E. Kaseb, Mostafa R. Fayoum Univ Fac Comp & Artificial Intelligence Dept Comp Sci Al Fayyum 63514 Egypt

Data growth in recent years has been swift, leading to the emergence of big data science. distributed File Systems (DFS) are commonly used to handle big data, like Google File System (GFS), Hadoop distributed File System (HDFS), and others. The DFS should provide the availability of data and reliability of the system in case of failure. The DFS replicates the files in different locations to provide availability and reliability. These replications consume storage space and other resources. The importance of these files differs depending on how frequently they are used in the system. So some of these files do not deserve to replicate many times because it is unimportant in the system. This paper introduces a Dynamic Replication Policy using Machine Learning Clustering (DRPMLC) on HDFS, which uses Machine Learning to cluster the files into different groups and apply other replication policies to each group to reduce the storage consumption, improve the read and write operations time and keep the availability and reliability of HDFS as a High-Performance distributed computing (HPDC).

关键词： Feature extraction File systems Machine learning Databases Big Data distributed computing Support vector machines Replicability Availability big data clustering Hadoop distributed file system high-performance distributed computing machine learning reliability replication policy

来源：评论

学校读者我要写书评

暂无评论

A lower bound for constant-size local certification

引用

THEORETICAL COMPUTER SCIENCE 2023年第1期971卷

作者： Martinez, Virginia Ardevol Caoduro, Marco Feuilloley, Laurent Narboni, Jonathan Pournajafi, Pegah Raymond, Jean-Florent Univ Paris 09 LAMSADE Paris France Univ Grenoble Alpes Lab G SCOP Grenoble France Univ Lyon CNRS INSA Lyon LIRISUMR5205UCBL Villeurbanne France Univ Bordeaux LaBRI CNRS Bordeaux France Univ Lyon ENS Lyon LIP UCBLCNRSLyon Lyon France Univ Clermont Auvergne CNRS LIMOS Clermont Ferrand France

Given a network property or a data structure, a local certification is a labeling that allows to efficiently check that the property is satisfied, or that the structure is correct. The quality of a certification is measured by the size of its labels: the smaller, the better. This notion plays a central role in self-stabilization, because the size of the certification is a lower bound (and often an upper bound) on the memory needed for silent self-stabilizing construction of distributed data *** it comes to the size of the certification labels, one can identify three important regimes: the properties for which the optimal size is polynomial in the number of vertices of the graph, the ones that require only polylogarithmic size, and the ones that can be certified with a constant number of bits. The first two regimes are well studied, with several upper and lower bounds, specific techniques, and active research questions. On the other hand, the constant regime has never been really explored. The main contribution of this paper is the first non-trivial lower bound for this constant regime. More precisely, we show that by using certification on just one bit (a binary certification), one cannot certify k-colorability for k & GE;3. To do so, we develop a new technique, based on the notion of score, and both local symmetry arguments and a global parity *** hope that this technique will be useful for establishing stronger results. We complement this result with an upper bound for a related problem, illustrating that in some cases one can do better than the natural upper bound.& COPY;2023 Elsevier B.V. All rights reserved.

关键词： Local certification Locally check -able labellings distributed computing Graph coloring Graph theory

来源：评论

学校读者我要写书评

暂无评论

Ensemble Classification With Noisy Real-Valued Base Functions

引用

IEEE JOURNAL ON SELECTED AREAS IN COMMUNICATIONS 2023年第4期41卷 1067-1080页

作者： Ben-Hur, Yuval Goren, Asaf Klang, Da-El Kim, Yongjune Cassuto, Yuval Technion Israel Inst Technol Dept Elect & Comp Engn IL-3200003 Haifa Israel Pohang Univ Sci & Technol POSTECH Dept Elect Engn Pohang 37673 South Korea Yonsei Univ Inst Convergence Res & Educ Adv Technol Seoul 03722 South Korea

In data-intensive applications, it is advantageous to perform partial processing close to the data, and communicate intermediate results to a central processor, instead of the data itself. When the communication or computation medium is noisy, the resulting degradation in computation quality at the central processor must be mitigated. We study this problem for the setup of binary classification performed by an ensemble of base functions communicating real-valued confidence levels. We propose a noise-mitigation solution that optimizes the transmission gains and aggregation coefficients of the base functions. Toward that, we formulate a post-training gradient-based optimization algorithm that minimizes the error probability given the training dataset and the noise parameters. We further derive lower and upper bounds on the optimized error probability, and show empirical results that demonstrate the enhanced performance achieved by our approach on real data.

关键词： Noise measurement Training Classification algorithms Optimization Reliability Performance evaluation Hardware Machine learning classification algorithms Index Terms boosting inference algorithms distributed computing Gaussian noise

来源：评论

学校读者我要写书评

暂无评论

Transmitting Once to Elect a Leader on Wireless Networks

引用

ALGORITHMICA 2023年第9期85卷 2529-2553页

作者： Ravelomanana, Vlady Andriambolamalala, Ny Aina Univ Paris Cite IRIF UMR CNRS 8243 Paris France

distributed wireless network devices are mostly battery-powered. Transmitting a message on distributed wireless networks uses more energy than receiving one, which in turn uses more energy than internal computations. Therefore, in this paper, we study the problem of randomized leader election in synchronous distributed single-hop networks with a special focus on the energy complexity. We provide algorithmic solutions to the implicit version of leader election problem where non-leader nodes need not be aware of the identity of the leader. Because the size of a message impacts the energy consumption, we highlight that the solutions we propose consume very little energy: each device is allowed to send a single one-bit message only once and listen to the network during two time slots at most. We first consider four well-studied variants of the radio network (RN) model depending on the transmission and reception abilities of the participating devices: - RNsTRoNGCD: both transmitters and listeners can detect collision,- RNsENDERCD: only transmitters can detect collision,- RNCD: only listeners can detect collision,- RNNoCD: no device can detect collision. Next, we study the beeping network model. The time and energy complexities of all our algorithms are deterministic and they succeed in electing a unique leader with high probability even under the restriction that each node can only send once a single one-bit signal. When the nodes are aware of the total number n of the participants, our algorithm elects a leader in O (log n) rounds. When n is not known beforehand but an upper bound u on n with log u = theta (log n) is known by all participating nodes, we design a randomized algorithm with O (log(2) n) time complexity for the RN models. For beeping networks, our algorithm has O (n(epsilon)) time complexity (0 < epsilon < 1). The parameter epsilon can be tuned to increase the probability of success of the algorithms.

关键词： distributed computing Leader election Energy complexity

来源：评论

学校读者我要写书评

暂无评论

sasa: a SimulAtor of Self-stabilizing Algorithms

引用

COMPUTER JOURNAL 2023年第4期66卷 796-814页

作者： Altisen, Karine Devismes, Stephane Jahier, Erwan Univ Grenoble Alpes Grenoble INP VERIMAG CNRS F-38000 Grenoble France Univ Picardie Jules Verne MIS F-80039 Amiens 1 France

In this paper, we present sasa, an open-source SimulAtor of Self-stabilizing Algorithms. Self-stabilization defines the ability of a distributed algorithm to recover after transient failures. sasa is implemented as a faithful representation of the atomic-state model (also called the locally shared memory model with composite atomicity). This model is the most commonly used one in the self-stabilizing area to prove both the correct operation of self-stabilizing algorithms and complexity bounds on them. sasa encompasses all features necessary to debug, test and analyze self-stabilizing algorithms. All these facilities are programmable to enable users to accommodate to their particular needs. For example, asynchrony is modeled by programmable stochastic daemons playing the role of input sequence generators. Properties of algorithms can be checked using formal test oracles. The sasa distribution also provides several facilities to easily achieve (batch-mode) simulation campaigns. We show that the lightweight design of sasa allows to efficiently perform huge such campaigns. Following a modular approach, we have aimed at relying as much as possible the design of sasa on existing tools, including ocaml, dot and several tools developed in the Synchrone Group of the VERIMAG laboratory.

关键词： simulation debugging reactive programs synchronous languages distributed computing self-stabilization atomic-state model

来源：评论

学校读者我要写书评

暂无评论

HCTTI: High-Performance Heterogeneous computing Toolkit for Tissue Image Stain Normalization

引用

JOURNAL OF IMAGING INFORMATICS IN MEDICINE 2025年 1-13页

作者： Jiang, Yan Wang, Bo Xing, Weipeng Huo, Cheng Quanzhou Univ Informat Engn Sch Software Quanzhou 362000 Fujian Peoples R China Geneis Beijing Co Ltd Sci Dept Beijing 100102 Peoples R China Changsha Med Univ Sch Informat Engn Changsha 410219 Peoples R China Anhui Univ Technol Sch Elect & Informat Engn Maanshan 243002 Anhui Peoples R China Sinopharm Tongmei Gen Hosp Dept Pathol Datong 037003 Shanxi Peoples R China

Whole slide imaging (WSI) has transformed diagnostic medicine, particularly in the field of cancer diagnosis and treatment. The use of deep learning algorithms for predicting WSIs has opened up new avenues for advanced medical diagnostics. Additionally, stain normalization can reduce the color and intensity variations present in WSI from different hospitals. As a result, deep learning classification accuracy improves. However, WSI reading and color normalization are still largely performed by using CPUs, leading to sub-optimal performance. We proposed a High-Performance Heterogeneous computing Toolkit for Tissue Image (HCTTI) that integrates multiple computer system-level optimizations and encompasses WSI reading, tile normalization, and tile saving. We explored the potential advantages and limitations of different WSI readers and color normalization techniques in WSI analysis and the performance of different tile serialization formats for saving tiles. We found that HCTTI is 7x\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\times $$\end{document} faster than OpenSlide for reading WSIs, GPU implementation of the Macenko normalization algorithm is 9x\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\times $$\end{document} faster than TIAToolbox implementation, and HDF5 is faster than png and Zarr for storing normalized images in both writing (13x\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\times $$\end{document} acceleration com

关键词： Histopathology image analysis distributed computing Stain normalization Digital pathology Open source software

来源：评论

学校读者我要写书评

暂无评论

Gathering Over Heterogeneous Meeting Nodes

引用

COMPUTER JOURNAL 2023年第5期67卷 1794-1813页

作者： Chakraborty, Abhinav Bhagat, Subhash Mukhopadhyaya, Krishnendu Siksha O Anusandhan Univ Inst Tech Educ & Res Dept Comp Sci & Engn Bhubaneswar 751030 Orissa India Indian Inst Technol Jodhpur Dept Math Jodhpur 342011 Rajasthan India Indian Stat Inst Adv Comp & Microelect Unit Kolkata 700108 West Bengal India

We consider two finite and disjoint sets of homogeneous robots deployed at the nodes of an infinite grid graph. The grid graph also comprises two finite and disjoint sets of prefixed meeting nodes located over the nodes of the grid. The objective of our study is to design a distributed algorithm that gathers all the robots belonging to the first team at one of the meeting nodes belonging to the first type, and all the robots in the second team must gather at one of the meeting nodes belonging to the second type. The robots can distinguish between the two types of meeting nodes. However, a robot cannot identify its team members. This paper assumes the strongest adversarial model, namely the asynchronous scheduler. We have characterized all the initial configurations for which the gathering problem is unsolvable. For the remaining initial configurations, the paper proposes a distributed gathering algorithm. Assuming the robots are capable of global-weak multiplicity detection, the proposed algorithm solves the problem within a finite time period. The algorithm runs in $\Theta (dn)$ moves and $O(dn)$ epochs, where $d$ is the diameter of the minimum enclosing rectangle of all the robots and meeting nodes in the initial configuration, and $n$ is the total number of robots in the system.

关键词： distributed computing mobile robot infinite grid asynchronous scheduler Look-Compute-Move Cycle gathering

来源：评论

学校读者我要写书评

暂无评论

Bloccess: Enabling Fine-Grained Access Control Based on Blockchain

引用

JOURNAL OF NETWORK AND SYSTEMS MANAGEMENT 2023年第1期31卷 6-6页

作者： Ding, Yepeng Sato, Hiroyuki Univ Tokyo Dept Elect Engn & Informat Syst Tokyo 1138654 Japan

Access control is a fundamental security aspect and has been adopted in diverse systems. Particularly, fine-grained access control models present good flexibility and scalability to adapt to complicated systems. However, building a trustworthy fine-grained access control mechanism in untrustworthy distributed environments such as Internet of Things (IoT) environments is challenging. Conventional access control mechanisms encounter security and privacy issues caused by centralized entities, such as single point of failure and data tampering. To address these issues, we have proposed Bloccess, a fine-grained access control framework based on the consortium blockchain, in our previous work. By leveraging blockchain technology, we formulate a set of protocols to enforce a tamper-proof access control mechanism in untrustworthy distributed environments. In this paper, we refine our previous work and present the extended version of Bloccess. We optimize our protocols and extend them to support a hybrid blockchain structure. We also formulate complete identification protocols for the administration mechanism in Bloccess. Besides, we show Bloccess in practice with a Bloccess-enabled IoT system. Furthermore, we conduct a semi-formal analysis to prove the security properties of Bloccess and evaluate its security through a security model and a threat model.

关键词： Access control Blockchain Security and privacy Formal methods Internet of things distributed computing Software Engineering

来源：评论

学校读者我要写书评

暂无评论

Learning and Batch-Processing Based Coded Computation With Mobility Awareness for Networked Airborne computing

引用

IEEE TRANSACTIONS ON VEHICULAR TECHNOLOGY 2023年第5期72卷 6503-6517页

作者： Wang, Baoqian Xie, Junfei Lu, Kejie Wan, Yan Fu, Shengli Univ Calif Dept Elect & Comp Engn San Diego CA 92093 USA San Diego State Univ Dept Elect & Comp Engn San Diego CA 92182 USA San Diego State Univ Dept Elect & Comp Engn San Diego CA 92182 USA Univ Puerto Rico Mayaguez Dept Comp Sci & Engn Mayaguez PR 00681 USA

The implementation of many Unmanned Aerial Vehicle (UAV) applications (e.g., fire detection, surveillance, and package delivery) requires extensive computing resources to achieve reliable performance. Existing solutions that offload computation tasks to the ground may suffer from long communication delays. To address this issue, the Networked Airborne computing (NAC) is a promising technique, which offers advanced onboard airborne computing capabilities by sharing resources among the UAVs via direct flight-to-flight links. However, NAC does not exist yet and enabling it requires overcoming many technical challenges, such as the high UAV mobility, and the uncertain, heterogeneous, and dynamic airspace. This paper addresses these challenges by 1) developing a Dynamic Batch-Processing based Coded Computation (D-BPCC) framework for achieving robust and adaptable cooperative airborne computing, and 2) designing deep reinforcement learning (DRL) based load allocation and UAV mobility control strategies for optimizing the system performance. As the first study to systematically investigate NAC, to the best of our knowledge, we evaluate the proposed methods through designing a NAC simulator and conducting comparative studies with four state-of-the-art distributed computing schemes. The results demonstrate the promising performance of the proposed methods.

关键词： Task analysis Autonomous aerial vehicles distributed computing Vehicle dynamics Trajectory Servers Computational modeling Networked airborne computing unmanned aerial vehicle coded distributed computing reinforcement learning load allocation mobility control

来源：评论

学校读者我要写书评

暂无评论

Decentralized Asynchronous Nonconvex Stochastic Optimization on Directed Graphs

引用

IEEE TRANSACTIONS ON CONTROL OF NETWORK SYSTEMS 2023年第4期10卷 1796-1804页

作者： Kungurtsev, Vyacheslav Morafah, Mahdi Javidi, Tara Scutari, Gesualdo Czech Tech Univ Dept Comp Sci Prague Czech Republic Univ Calif San Diego Dept Elect Engn La Jolla CA 92093 USA Purdue Univ Sch Ind Engn W Lafayette IN 47907 USA

In this article, we consider a decentralized stochastic optimization problem over a network of agents, modeled as a directed graph: Agents aim to asynchronously minimize the average of their individual losses (possibly nonconvex), each one having access only to a noisy estimate of the gradient of its own function. We propose an asynchronous distributed algorithm for such a class of problems. The algorithm combines stochastic gradients with tracking in an asynchronous push-sum framework and obtains a sublinear convergence rate, matching the rate of the centralized stochastic gradient descent applied to the nonconvex minimization. Our experiments on a nonconvex image classification task using a convolutional neural network validate the convergence of our proposed algorithm across a different number of nodes and graph connectivity percentages.

关键词： Optimization Stochastic processes Convergence Delays Directed graphs Noise measurement Linear programming Decentralized applications distributed computing federated learning machine learning optimization optimization methods

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：