检索结果-内蒙古大学图书馆

Towards Close-to-Zero Runtime Collection Overhead: Raft-Based Anomaly Diagnosis on system Faults for distributed storage system

引用

IEEE TRANSACTIONS ON SERVICES COMPUTING 2025年第2期18卷 1054-1067页

作者： Zhang, Lingzhe Jia, Tong Jia, Mengxi Liu, Hongyi Yang, Yong Wu, Zhonghai Li, Ying Peking Univ Beijing 100871 Peoples R China

distributed storage systems are fundamental infrastructures of today's large-scale software systems such as cloud systems. Diagnosing anomalies in distributed storage systems is essential for maintaining software availability. Existing anomaly diagnosis approaches mainly rely on the run-time data including monitoring data and application logs. However, collecting and analyzing the run-time data requires huge computing, storage, and management costs. Typically, more fine-grained run-time data can reveal more symptoms of anomalies, but on the contrary, requires more computing, storage, and management costs. As a result, solving the anomaly diagnosis problem is a balancing between the quality of run-time data and system overhead or cost. In this paper, we take into account both data quality and system overhead or cost by introducing a new type of run-time data-Raft logs. Raft logs are naturally produced by distributed storage systems and collecting raft logs will not bring any extra system overhead. To verify the ability of Raft logs in reflecting anomalies, we conduct a comprehensive study on the interconnection between the anomalies and Raft logs. Based on the study, we propose an effective Raft-Based Anomaly Diagnosis approach named RBAD. For evaluation, we expose the first open-sourced comprehensive dataset with multiple runtime data containing both Raft logs, application logs and monitoring data. Experiments based on this dataset demonstrate RBAD's superiority, outperforming monitoring-based methods by 15.38% and log-based methods by 53.10%.

关键词： Monitoring distributed databases Feature extraction Databases Costs Runtime Measurement Data mining Accuracy Vectors Anomaly diagnosis distributed storage system raft

来源：评论

学校读者我要写书评

暂无评论

Integrita: A BFT distributed storage system

引用

FUTURE GENERATION COMPUTER systemS-THE INTERNATIONAL JOURNAL OF ESCIENCE 2025年 166卷

作者： Boshrooyeh, Sanaz Taheri Kupcu, Alptekin Ozkasap, Oznur Koc Univ Dept Comp Engn Istanbul Turkiye

Collaborative data sharing underlies applications in systems such as online social networks and cloud storage. A central provider hosts shared data, e.g., a Facebook group page, and provides sharing users with read/write access according to user-defined settings. Historical incidents prove that data storage centralization enables a corrupted provider to censor or diverge users' views of the shared data. Such misbehavior is hardly detectable as users update the shared data unbeknown to each other. Existing solutions suggest server-side data replication, which is storage inefficient, or users' out-of-band communication, which is communication intensive. Addressing these inefficiencies, we propose Integrita, a data-sharing mechanism that protects users' view-consistency needless to out-of-band communication. We present a novel distributed history tree algorithm to partition the shared data among N storage providers, N - 1 of which are Byzantine faulty and colluding. Our data partitioning solution reduces storage overhead by a multiplicative factor of N and allows provable detection of server-side equivocation and identification of corrupted servers. We introduce and achieve anew consistency level, named q-detectable consistency, where users' views inconsistency cannot remain undetected for more than q updates, q being the system parameters' function. Without loss of generality, we use online social networks as a case study to deploy Integrita and supply performance/numerical analysis accordingly.

关键词： View consistency q-detectable consistency Strong consistency Censorship resistant Malicious adversary Byzantine fault tolerant Collaborative data sharing History integrity distributed storage system Log-based system

来源：评论

学校读者我要写书评

暂无评论

Energy-efficient algorithms for distributed storage system based on block storage structure reconfiguration

引用

JOURNAL OF NETWORK AND COMPUTER APPLICATIONS 2015年 48卷 71-86页

作者： Liao, Bin Yu, Jiong Zhang, Tao Guo Binglei Hua, Sun Ying, Changtian Xinjiang Univ Finance & Econ Coll Stat & Informat Urumqi 830012 Peoples R China Xinjiang Univ Sch Software Urumqi 830008 Peoples R China Xinjiang Med Univ Dept Med Engn & Technol Urumqi 830011 Peoples R China

As the underlying core infrastructure for cloud computing, distributed storage systems like the Hadoop distributed File system (HDFS) are the foundation of all kinds of cloud services. However, designers of the ever-expanding systems have ignored the problem of high energy consumption, causing serious environmental and economic problems. The data availability and performance Quality of Service (QoS) requirements make it hard to use existing energy-saving technologies to solve the problem. After researching the data block's storage structure and mechanism, and the relationship between the server's status and the data block's availability, the method to solve the problem of ensuring data availability and performance QoS requirements is proposed. The energy-saving model for the distributed storage system is defined. The algorithm divides the RACK into two distinct storage areas, Active-Zone and Sleep-Zone, reconfiguring the data storage structure using the block storage structure reconfiguration algorithm. To save energy, we turn the servers in Sleep-Zone to sleep mode while the workload is low. Numerical analysis and experimental results demonstrate that the energy-efficient algorithms improved the energy efficiency for the distributed storage system. (C) 2014 Elsevier Ltd. All rights reserved.

关键词： Cloud computing distributed storage system Green computing QoS constraints

来源：评论

学校读者我要写书评

暂无评论

Flimm: Foreground traffic aware data migration manager for distributed storage system

引用

FUTURE GENERATION COMPUTER systemS-THE INTERNATIONAL JOURNAL OF ESCIENCE 2024年 160卷 140-153页

作者： Li, Bowen Dong, Xiaoshe Mi, Jue Wang, Yufei Wang, Longxiang Chen, Weiduo Xi An Jiao Tong Univ Sch Comp Sci & Technol 28 Xianning West Rd Xian Peoples R China State Grid Shaanxi Elect Power Co Informat & Commun Co Xian Peoples R China

Currently, decentralized distributed storage system is widely used in various scenarios, because decentralized data placement algorithms observably improve scalability and robustness of distributed storage system. However, storage system applied with decentralized data placement algorithm suffers from uncontrollable data migration during expanding the clusters, which lead to dramatic fluctuation of performance. To maintain reliability and stability of storage system, administrators are under great pressure to migrate data during expansion or contraction. Not only the urgent time constraints to expansion but also low interference to foreground application are required during data migration period. It is a difficult challenge for in-experienced administrators to tackle. To address this challenge, this paper introduces Flimm, a fast and low-interference data migration manager that is aware of storage resource utilization and performs data migration with low interference to foreground applications. Flimm mitigates interference by scheduling a subset of migration tasks with careful consideration. Moreover, Flimm adapts to varying foreground application workloads through adaptive migration speed control, providing administrators with convenient and secure parameters for tuning. In our experiments, Flimm demonstrated a 1.2x acceleration in migration speed and achieved a 20% reduction in interference.

关键词： distributed storage system Data migration Schedule algorithm Interference model Portable system

来源：评论

学校读者我要写书评

暂无评论

A method of repairing single node failure in the distributed storage system based on the regenerating-code and a hybrid genetic algorithm

引用

NEUROCOMPUTING 2021年 458卷 566-578页

作者： Ye, Miao Qiu, Hongbing Wang, Yong Zhou, Zou Zheng, Fei Ma, Tianxin Guilin Univ Elect Technol Sch Informat & Commun 1 Jinji Rd Guilin 541004 Peoples R China Beijing Normal Univ Dept Phys 19 XinJieKouWai St Beijing 100875 Peoples R China Guilin Univ Elect Technol Minist Educ Key Lab Cognit Radio & Informat Proc 1 Jinji Rd Guilin 541004 Peoples R China 54th Res Inst China Elect Technol Grp Corp 589 West Zhongshan Rd Shijiazhuang 050081 Hebei Peoples R China

To ensure the reliability and security of the data, the large-scale distributed storage system usually adopts the data redundancy mechanism to repair the data on the faulty nodes. Comparing with the replication-based method, the data redundancy mechanism of erasure code can effectively improve the use of storage space, but it may result in the large network overhead when recovering the data. The regenerating code is an improved erasure code, which can reduce the quantity of data transmission compared to that of the erasure code. Adopting the regenerating code to repair the data on a faulty node requires constructing an optimal repair tree to maximum the bandwidth of the bottleneck link, which is an NP-hard problem. To construct the optimal repair tree, a hybrid genetic algorithm is proposed in this paper. In particular, our proposal comprehensively considers the network topology and link bandwidth of storage nodes and designs a problem-specific cross-correlation operator, mutation operator and local search operator. In addition, we provided the mathematical proof of the global convergence with probability one with respect to the proposed hybrid genetic algorithm. Through a series of simulation experiments, the results show that our proposal is able to determine the optimal repair tree, which effectively reduces the delay of faulty node repair in distributed storage systems, and improve the repairing efficiency. CO 2020 Published by Elsevier B.V.

关键词： distributed storage system Erasure code Regenerating code Hybrid Genetic algorithm Optimal repair tree

来源：评论

学校读者我要写书评

暂无评论

Async-LCAM: a lock contention aware messenger for Ceph distributed storage system

引用

CLUSTER COMPUTING-THE JOURNAL OF NETWORKS SOFTWARE TOOLS AND APPLICATIONS 2019年第2期22卷 373-384页

作者： Jeong, Bodon Khan, Awais Park, Sungyong Sogang Univ Dept Comp Sci & Engn 35 Baekreom Ro Seoul South Korea Sogang Univ Dept Comp Sci & Engn Lab Adv Syst Software 35 Baekreom Ro Seoul South Korea

Ceph, an object-based distributed storage system, has a communication subsystem called Async messenger. In the Async messenger, a worker thread in the thread pool is assigned to each connection in a round-robin fashion and is allowed to process all the incoming or outgoing messages from the connection. Although this thread per connection strategy is easy to implement, it has an inherent problem such that when a connection is overloaded, it results in load imbalance problem among worker threads. In order to mitigate this problem, multiple worker threads can be assigned to a single connection to handle the traffic from the connection. However, this mapping structure induces another overhead related to lock contention since multiple threads contend to access the shared resources in the connection. In this paper, we propose lock contention aware messenger (Async-LCAM) , a messenger that assigns multiple worker threads per connection and is aware of lock contention generated from the threads. By keeping track of the lock contention of each connection every interval, the Async-LCAM dynamically adds or deletes assigned threads to/from the connection in order to balance the workloads among worker threads. The experimental results show that the Async-LCAM improves the throughput and latency of Ceph storage by up to 184 and 65%, respectively, compared to the original Async messenger.

关键词： distributed storage system Ceph Thread placement Load balancing Lock contention Epoll

来源：评论

学校读者我要写书评

暂无评论

Network Coding for Energy-Efficient distributed storage system in Wireless Sensor Networks

引用

KSII TRANSACTIONS ON INTERNET AND INFORMATION systemS 2013年第9期7卷 2134-2152页

作者： Wang Lei Yang Yuwang Zhao Wei Lu Wei Nanjing Univ Sci & Technol Sch Comp Sci Nanjing Jiangsu Peoples R China

A network coding-based scheme is proposed to improve the energy efficiency of distributed storage systems in WSNs (wireless sensor networks), which mainly focuses on two problems: firstly, consideration is given to effective distributed storage technology in WSNs;secondly, we address how to repair the data in failed storage nodes with less resource. For the first problem, we propose a method to obtain a sparse generator matrix to construct network codes, and this sparse generator matrix is proven to be the sparsest. Benefiting from the sparse generator matrix, the energy consumption required to implement distributed storage is reduced. For the second problem, we designed a network coding-based iterative repair method, which adequately utilizes the idea of re-encoding at intermediate nodes from network coding theory. Benefiting from the re-encoding, the energy consumption required by data repair is significantly reduced. Moreover, we provide an explicit lower bound of field size required by this scheme, which implies that this scheme can work over a very small field and the required computation overhead of coding is very low. The simulation result verifies that by using our scheme, the total energy consumption required to implement distributed storage system in WSNs can be reduced on the one hand, and on the other hand, this method can also balance energy consumption of the networks.

关键词： network coding distributed storage system MDS wireless sensor network energy efficiency

来源：评论

学校读者我要写书评

暂无评论

Model of a distributed storage system for Crypto Wallet Private Keys

引用

AUTOMATIC CONTROL AND COMPUTER SCIENCES 2024年第8期58卷 1289-1296页

作者： Salikhov, M. R. ITMO Univ St Petersburg 191002 Russia

With the development of Web3 technologies, the third generation of the Internet has become one of the most promising areas. It involves the use of decentralized, transparent, and user-oriented applications. However, many Web3 projects do not pay sufficient attention to security, which can lead to serious consequences. Even a small error in the code can make the system vulnerable, opening access to attackers. As a result, the industry faces frequent security breaches that threaten users and undermine trust in new technologies. One of the main problems with Web3 is private key management. This is a critical security aspect that is directly related to the protection of digital assets and users' personal information. The risk of losing or theft of a private key can lead to irreparable consequences, since in the case of loss there is no way to restore or reset the key. Various ways of storing the private key of a crypto wallet to ensure security are discussed. For example, the key can be split into parts and stored encrypted on hardware media, or the entire encrypted key can be stored on secure media. Quantitative data are calculated using Shamir's scheme.

关键词： key management encryption secret sharing cryptography distributed storage system

来源：评论

学校读者我要写书评

暂无评论

Load Balancing Technology based on Erasure Code in distributed storage system

Load Balancing Technology based on Erasure Code in Distribut...

引用

31st Chinese Control Conference

作者： Ding Caiwei Wang Lei Sun Liansheng USTC Dept Automat Key Lab Network Commun Syst & Control Hefei 230027 Peoples R China

ISBN: (纸本)9789881563811

LandHouse is a distributed storage system that is made up of some regular graphs and is organized by distributed hash table. However, the master server is the performance bottleneck of LandHouse. Moreover, when using traditional data block strategy, the regular graph Petersen can't work to its best extent. Therefore, a new architecture with modified node deployment is designed to optimize the original system. Besides, a novel data block strategy named FTF using erasure code technology and an improved dynamic load balancing algorithm on Petersen Ring are proposed. Simulation results show that these modifications can improve robustness and performance of the system effectively.

关键词： LandHouse load balance petersen erasure code distributed storage system

来源：评论

学校读者我要写书评

暂无评论

A Data Migration Scheme Considering Node Reliability for an Autonomous distributed storage system 5

A Data Migration Scheme Considering Node Reliability for an ...

引用

5th ACIS International Conference on Computational Science/Intelligence and Applied Informatics (CSII)

作者： Nunome, Atsushi Hirata, Hiroaki Kyoto Inst Technol Fac Informat & Human Sci Kyoto Japan

ISBN: (纸本)9781538678756

We have proposed a distributed storage system which aims to utilize storage fragments distributed in network nodes without administrators' maintenance cost. Even the performance of storage devices are not uniform, autonomously data block migration helps to achieve enhancing the whole I/O performance of the storage system. However, we have considered all the storage devices to be reliable equally. On this assumption, migration of an important data block to unstable storage might cause degradation of the availability of the data block. In this paper, we propose a scheme to choose the destination of data block migration by evaluating the nodes' reliability along with the performance of storage devices. This scheme aims to improve system stability by preventing to locate a critical data onto an unstable storage device. The storage nodes which lack stability are also utilized for storing a data block which can be easily regenerated or duplicated. The results of the preliminary investigation show that a large number of warning logs from Machine Check Architecture may suggest a sign of unexpected system shutdown.

关键词： distributed storage system system reliability storage tiering block relocation

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：