With the rapid development of storage and network technology, emerging high-performance hardware is being widely applied to the distributed storage cluster. However, existing distributed storage systems employing mult...
详细信息
ISBN:
(数字)9798350317152
ISBN:
(纸本)9798350317169
With the rapid development of storage and network technology, emerging high-performance hardware is being widely applied to the distributed storage cluster. However, existing distributed storage systems employing multi-layer abstractions to provide table data services result in leaving high-speed hardware under-exploited. In this paper, we propose TEngine, a native distributed table storage engine designed for NVMe SSD and RDMA. The key is that TEngine removes the file abstraction to construct table structures on the device directly. For metadata service, TEngine designs a decoupled single metadata server, reducing distributed coordination, easing the burden on the metadata node, and enabling localized data node access. For data service, TEngine optimizes the parallel processing capability of NVMe devices by integrating upper-level multi-thread parallel operations with lower-level NVMe devices' parallel I/O processing. Moreover, TEngine introduces a periodic pull-based data synchronization approach to transform data pushing into periodic data pulling, which offloads the synchronization burden from the leader to the followers. The experimental results show that TEngine outperforms state-of-the-art distributed storage systems using the same hardware environment.
Web applications rely on database systems to store and manage data. Existing major databases use lock-based concurrency control to coordinate transactions. However, lock-based transaction processing can be expensive a...
详细信息
Web applications rely on database systems to store and manage data. Existing major databases use lock-based concurrency control to coordinate transactions. However, lock-based transaction processing can be expensive and result in unnecessary deadlocks because the database systems are oblivious to the application semantics, such as access patterns. This paper examines an issue commonly seen in web applications: inefficient associated accesses. They are problematic because they cause database systems to abort concurrent transactions frequently, degrading application performance. To address this issue, we build Railyzer, a tool that automatically analyzes web applications’ database access patterns and provides possible optimization strategies. First, we establish a criterion to differentiate associated accesses from other accesses. Then, based on a language-agnostic analysis of the application, we create a method for identifying associated accesses. Finally, we use heuristics to locate inefficient ones and suggest fixes. We discovered 83 potential optimizations among the six open source applications. Some of these optimizations improve the application throughput by up to 70%.
The end of Moore's law has placed a two-fold demand on hardware simulation. Firstly, efficient co-design requires fast simulation of hardware systems in order to vet proposed designs. Secondly, modern simulator pl...
详细信息
This paper overcomes the shortcomings of the existing technology, and proposes a trust calculation method of Internet of things for frequency modulation transactions of West Power to East power transmission units. The...
详细信息
ISBN:
(数字)9798350352719
ISBN:
(纸本)9798350352726
This paper overcomes the shortcomings of the existing technology, and proposes a trust calculation method of Internet of things for frequency modulation transactions of West Power to East power transmission units. The basic principle is as follows: Considering the data information formed by data transmission rate and data storage and sharing, and the data error and missing formed by data collection, the iot trust degree for frequency modulation transaction of West-East Power transmission unit is calculated by using the sufficient amount and missing amount of information, and the influence of trust degree on frequency modulation transaction of West-East power transmission unit is evaluated.
This paper proposes a method for generating the surface model of hollow turbine blades based on the fusion of Industrial Computerized Tomography (ICT) multi-directional slice data. Through the Normal Distribution Tran...
详细信息
ISBN:
(数字)9798350352719
ISBN:
(纸本)9798350352726
This paper proposes a method for generating the surface model of hollow turbine blades based on the fusion of Industrial Computerized Tomography (ICT) multi-directional slice data. Through the Normal Distribution Transformation (NDT) algorithm and the Iterative Closest Points (ICP) algorithm, the registration fusion of CT multi-directional slice data is achieved, obtaining turbine blade point cloud data containing more comprehensive key structural features. The Delaunay triangulation algorithm is used for surface model generation, obtaining a detailed model of the entire inner and outer surfaces of the hollow turbine blades.
With the attractive characteristics of scalability, strong consistency, and high availability, distributeddatabases have attracted much attention. Moreover, application-oriented database development promotes the fast...
详细信息
We present a scalable parallel I/O system for a logical-inferencing application built atop a deductive database. Deductive databases can make logical deductions (i.e. conclude additional facts), based on a set of prog...
详细信息
ISBN:
(纸本)9781665435772
We present a scalable parallel I/O system for a logical-inferencing application built atop a deductive database. Deductive databases can make logical deductions (i.e. conclude additional facts), based on a set of program rules, derived from facts already in the database. Datalog is a language or family of languages commonly used to specify rules and queries for a deductive database. Applications built using Datalog can range from graph mining (such as computing transitive closure or k-cliques) to program analysis (control and data-flow analysis). In our previous papers, we presented the first implementation of a data-parallel Datalog built using MPI. In this paper, we present a parallel I/O system used to checkpoint and restart applications built on top of our Datalog system. State of the art Datalog implementations, such as Souffle, only support serial I/O, mainly because the implementation itself does not support many-node parallel execution. Computing the transitive closure of a graph is one of the simplest logical-inferencing applications built using Datalog;we use it as a micro-benchmark to demonstrate the efficacy of our parallel I/O system. Internally, we use a nested B-tree data-structure to facilitate fast and efficient in-memory access to relational data. Our I/O system therefore involves two steps, converting the application data-layout (a nested B-tree) to a stream of bytes followed by the actual parallel I/O. We explore two popular I/O techniques POSIX I/O and MPI collective I/O. For extracting performance out of MPI Collective I/O we use adaptive striping, and for POSIX I/O we use file-per-process I/O. We demonstrate the scalability of our system at up to 4,096 processes on the Theta supercomputer at the Argonne National Laboratory.
Forward Provenance for streaming queries run by distributed and parallel Stream Processing Engines gives fine-grained insights on input-output data dependencies enabling, e.g., precise debugging and smart data selecti...
详细信息
ISBN:
(数字)9798350386059
ISBN:
(纸本)9798350386066
Forward Provenance for streaming queries run by distributed and parallel Stream Processing Engines gives fine-grained insights on input-output data dependencies enabling, e.g., precise debugging and smart data selection. State-of-the-art provenance frameworks, though, build on an assumption that is unrealistic for distributedsystems like Vehicular Networks and Smart Grids, namely, that the whole set of queries in need of provenance is known in advance and static. In real-world use cases, queries are continuously added, removed, and modified over time by both data analysts and SPE systems themselves. Motivated by the lack of solutions for the forward provenance of dynamic sets of queries, we introduce a novel framework, named Nona, for parallel and distributed streaming queries. We formalize the notion of forward provenance for evolving query sets and prove it is possible to extend the same guarantees the state-of-the-art offers for static query sets. Our evaluation shows that Nona can cope with adaptations to changes in query sets with sub-second responsiveness; moreover, it incurs negligible overheads compared to the state-of-the-art, during the periods in which a query set does not undergo changes.
NVIDIA’s H100 Confidential Computing (CC) counters the security hazards inherent in cloud AI workloads. It enforces data encryption to achieve data confidentiality, which leads to substantial throughput reductions as...
详细信息
ISBN:
(数字)9798331509712
ISBN:
(纸本)9798331509729
NVIDIA’s H100 Confidential Computing (CC) counters the security hazards inherent in cloud AI workloads. It enforces data encryption to achieve data confidentiality, which leads to substantial throughput reductions as high as 93% in various AI workloads (such as TensorRT, PEFT and vLLM). Confronting this substantial overhead issue, we first delve into the underlying causes through meticulous analysis. This groundwork enables us to devise an innovative runtime system that operates seamlessly in the background, completely transparent to end-users. The cornerstone of our system lies in leveraging multiple encryption workers. Experiments demonstrate that our solution effectively reduces throughput drop to less than 28.1%.
The design of distributed database has became demanding with the increase in use of IoT and cloud based services. distributed database system's performance is totally relies on its design. Allocation of data is on...
详细信息
ISBN:
(纸本)9781665424615
The design of distributed database has became demanding with the increase in use of IoT and cloud based services. distributed database system's performance is totally relies on its design. Allocation of data is one of the major design issues while designing distributeddatabases. This paper presents a new technique for non-redundant allocation of data in distributed database design. The proposed approach allocates the data based on Simplified Biogeography Based Optimization (Simplified-BBO). The performance comparison of Simplified-BBO based approach is done against the GA and BBO based approaches. The proposed approach helps in decreasing the data communication cost during query execution which results in increasing the overall performance of distributed database systems.
暂无评论