检索结果-内蒙古大学图书馆

12th International Conference on Fuzzy Systems and Knowledge Discovery (FSKD)

作者： Li, Chaoyong Cheng, Gong Zhong, Jinwen Ma, Can Wang, Weiping Meng, Dan Wang, Qing Wang, Bo Chinese Acad Sci Inst Informat Engn Beijing 100093 Peoples R China Univ Chinese Acad Sci Beijing 100049 Peoples R China Natl Comp Network Emergency Response Tech Team Ct Beijing 100029 Peoples R China

ISBN: (纸本)9781467376822

With the size of data increasing continuously, a huge demand for interactive query on massive datasets emerges. When processing massive structured data, the existing query engines are lacking in utilizing the query locality and catching the difference among query operators, which results in their not being applied to low-latency business scenarios. To solve these problems, this paper proposes a new framework named Lotus for query optimization based on distributed cache. Lotus adopts three strategies: (1) performing query-sensitive data distribution policy;(2) carrying out cache replacement based on statistical information;(3) optimizing the behavior of core operators. Through the above methods, Lotus improves the query performance of existing engines. The experimental study shows that Lotus can reduce the response latency and execution time of queries on large-scale structured data by more than 30% in comparison with SparkSQL or Impala.

关键词： distributed cache interactive query data locality structured data

来源：评论

学校读者我要写书评

暂无评论

A distributed cache Mechanism of HDFS to Improve Learning Performance for Deep Reinforcement Learning 20

A Distributed Cache Mechanism of HDFS to Improve Learning Pe...

引用

20th IEEE Int Symposium on Parallel and distributed Processing with Applicat / 15th IEEE Int Conf on Social Comp and Networking / 12th IEEE Int Conf on Big Data and Cloud Comp / 12th IEEE Int Conf on Sustainable Comp and Commun

作者： Gao, Yongqiang Deng, Shunyi Li, Zhenkun Inner Mongolia Univ Coll Comp Sci Hohhot Peoples R China

ISBN: (纸本)9781665464970

Hadoop distributed file system (HDFS) has been widely used in deep reinforcement learning due to its high reliability and high performance. However, the dataset for training deep reinforcement learning models is usually composed of a large number of small files, which will cause the severe performance degradation of HDFS. Although many methods have been proposed to address small files problem in HDFS, none of them considers correlations between small files for training deep reinforcement learning models. To improve the learning performance for deep reinforcement learning, this paper proposes a new HDFS architecture, which uses a distributed cache mechanism to improve data access efficiency for training deep reinforcement learning models. The experimental results show that the proposed HDFS with distributed cache mechanism can achieve better learning performance compared to the traditional HDFS without cache and with centralized caching mechanism.

关键词： Deep reinforcement learning distributed cache HDFS

来源：评论

学校读者我要写书评

暂无评论

Design and implementation of distributed cache system in cluster environment 4

Design and implementation of distributed cache system in clu...

引用

4th International Conference on Advanced Design and Manufacturing Engineering, ADME 2014

作者： Huang, Yuan Qiang CNPC Research Institute of Safety and Environment Technology Beijing China

ISBN: (纸本)9783038352570

Within the layered enterprise application architecture, database is usually the bottleneck of system. cache technology can significantly improve the system performance and scalability by caching the data in application layer. Firstly, this paper discussed the significance and importance of the distributed cache System. Our system provided two kinds of cache mechanism, including replicated cache and partitioned cache. Each of these has advantages and disadvantages. The design and implementation of this system covered problems including consistent hashing arithmetic. This paper described and analyzed data distribution, data update and the procedure of operations in different cache mechanisms. At last, we made the compared test between our system and the similar product, Memcached. We found that there are both advantages and disadvantages of these two products, it points out the direction for our future improvement. © (2014) Trans Tech Publications, Switzerland.

关键词： Cluster Consistent Hashing distributed cache

来源：评论

学校读者我要写书评

暂无评论

distributed cache memory data migration strategy based on cloud computing

引用

CONCURRENCY AND COMPUTATION-PRACTICE & EXPERIENCE 2019年第10期31卷

作者： Wang Dongsheng Huang Chuanhe Wuhan Univ Comp Sch Wuhan 430072 Hubei Peoples R China Collaborat Innovat Ctr Geospatial Technol Wuhan 430072 Hubei Peoples R China

In view of the huge scale of cloud computing, cloud computing systems face many problems in terms of scalability and high availability. In view of the distributed data storage in the cloud computing environment, a recursion based N boundary network model is constructed. A data management model is given on the basis of the data center network structure. A replica distribution strategy and a copy selection strategy are designed to improve the data availability and load balance. On the premise of guaranteeing data availability, a data migration algorithm based on coverage set is proposed, which uses node selection strategy to reduce the migration cost as much as possible. Data migration makes more machines dormant and reduces energy consumption. By comparing and analyzing the experimental data, the correctness and effectiveness of the proposed network topology, data management model, and data migration technology are verified. In the edge cloud computing architecture, in order to ensure the response time of the cloud computing service and minimize the data redundancy in the system, this paper proposes and analyzes the data migration strategy and the pre stored data migration strategy for the network transmission, and gives the suitable application scene. On this basis, in order to control the balance of access network transmission and storage flexibly, a data migration strategy based on network performance is proposed, which ensures the real-time response of the service.

关键词： cloud computing data center network structure data migration distributed cache massive data

来源：评论

学校读者我要写书评

暂无评论

Three levels of fail-safe mode in MPI I/O NVRAM distributed cache

引用

Procedia Computer Science 2018年 136卷 52-61页

作者： Artur Malinowski Paweł Czarnul Dept. of Computer Architecture Faculty of Electronics Telecommunications and Informatics Gdańsk University of Technology Gdansk Poland

The paper presents architecture and design of three versions for fail-safe data storage in a distributed cache using NVRAM in cluster nodes. In the first one, cache consistency is assured through additional buffering write requests. The second one is based on additional write log managers running on different nodes. The third one benefits from synchronization with a Parallel File System (PFS) for saving data into a new file which allows to keep file history at the cost of space. We have shown that the three level fail-safe mode incorporating these versions does introduce minimal overhead for a random walk microbenchmark application for a 1GB file and checkpoints created every 2000 iterations, computing powers of a graph with 10000 vertices and up to 20% overhead for parallel processing of images up to 1000 megapixels compared to the basic NVRAM cache without fail-safe modes. We also presented times for checkpoint creation and restoring for sizes up to 10GBs.

关键词： NVRAM parallel MPI I/O extension fail safe distributed cache

来源：评论

学校读者我要写书评

暂无评论

HOTSPOT SENSITIVE DYNAMIC SCALING FOR distributed cache SYSTEMS

引用

INTERNATIONAL JOURNAL OF INNOVATIVE COMPUTING INFORMATION AND CONTROL 2019年第1期15卷 79-90页

作者： Zhou, Bo Li, Yaqiong Liu, Yongbo Li, Shouchao Song, Yunkui Jiangsu Hoperun Software Co 168 Ruanjian Ave Nanjing 210012 Jiangsu Peoples R China Chinese Acad Sci Inst Software 4 South Fourth St Beijing 100190 Peoples R China

distributed cache systems have been widely used to improve the applications' performance of accessing data by moving data and applications together in a cluster. However, existing works cannot well rebalance data partitions, and guarantee the data consistency and service availability during expanding cache systems. This paper proposes a hotspot sensitive dynamic scaling approach for distributed cache systems. First, we propose a hotspot sensitive data rebalance method, which is suitable for heterogeneous environment. It considers both memory utilization and network traffic, identifies hotspot data partitions, and then sets a high priority for the cache nodes with light workloads to guarantee the balance of cache nodes. Then, we propose a data accessing method based on two-phase controlled data migration to ensure the data consistency and service availability during scaling up cache systems. Finally, we have implemented a cache framework cacheScale, and conducted a series of experiments to validate the approach. The experimental results demonstrate that our approach can dynamically scale up cache systems, guarantee the data consistency and service availability, and achieve shorter response time.

关键词： distributed cache Dynamic scaling Hotspot data partition Data migration

来源：评论

学校读者我要写书评

暂无评论

distributed NVRAM cache - Optimization and Evaluation with Power of Adjacency Matrix 16th

Distributed NVRAM Cache - Optimization and Evaluation with P...

引用

16th IFIP TC8 International Conference on Computer Information Systems and Industrial Management (CISIM)

作者： Malinowski, Artur Czarnul, Pawel Gdansk Univ Technol Fac Elect Telecommun & Informat Dept Comp Architecture Gdansk Poland

ISBN: (纸本)9783319591056;9783319591049

In this paper we build on our previously proposed MPI I/O NVRAM distributed cache for high performance computing. In each cluster node it incorporates NVRAMs which are used as an intermediate cache layer between an application and a file for fast read/write operations supported through wrappers of MPI I/O functions. In this paper we propose optimizations of the solution including handling of write requests with a synchronous mode, additional modes preventing data preloading from a file and synchronization on file close if the solution is used as temporary cache only. Furthermore, we have evaluated the solution for a real application that computes powers of an adjacency matrix of a graph in parallel. We demonstrated superiority of our solution compared to a regular MPI I/O implementation for various powers and numbers of graph nodes. Finally, we presented good scalability of the solution for more than 600 processes running on a large HPC cluster.

关键词： NVRAM distributed cache Graph processing Performance optimization

来源：评论

学校读者我要写书评

暂无评论

A SOLUTION TO IMAGE PROCESSING WITH PARALLEL MPI I/O AND distributed NVRAM cache

引用

SCALABLE COMPUTING-PRACTICE AND EXPERIENCE 2018年第1期19卷 1-14页

作者： Malinowski, Artur Czarnul, Pawel Gdansk Univ Technol Fac Elect Telecommun & Informat Gdansk Poland

The paper presents a new approach to parallel image processing using byte addressable, non-volatile memory (NVRAM). We show that our custom built MPI I/O implementation of selected functions that use a distributed cache that incorporates NVRAMs located in cluster nodes can be used for efficient processing of large images. We demonstrate performance benefits of such a solution compared to a traditional implementation without NVRAM for various sizes of buffers used to read image parts, process and write back to storage. We also show that our implementation benefits from overlapping reading subsequent images while processing already loaded ones. We present results obtained in a cluster environment for three parallel implementation of blur, multipass blur and Sobel filters, for various NVRAM parameters such as latencies and bandwidth values.

关键词： image processing high performance computing NVRAM distributed cache Sobel blur filter

来源：评论

学校读者我要写书评

暂无评论

distributed caching strategy for hot news propagation based on data engineering processing

引用

INTERNET TECHNOLOGY LETTERS 2025年第1期8卷

作者： Duan, Lanlan Henan Light Ind Vocat Coll Zhengzhou Peoples R China

In order to improve the performance of the hot news collection and propagation of hotspot news, we aim at improving the performance and solve the opinion collection and propagation bottleneck by analyzing the demand and framework of current hot news supported systems. Then, a corresponding opinion collection and propagation optimization framework and strategy are proposed with a distributed hotspot caching system is designed and implemented. On the basis of the existing distributed search engine, a cache server is added between the overall hotspot query processing server and each cluster sub-query processing server, so that we can improve the caching and searching efficiency greatly. The experimental results show that after using the proposed distributed caching system and strategy for hot news propagation, the processing capacity of the hot news system has been greatly enhanced.

关键词： data engineering distributed cache hot news collection news propagation searching engine

来源：评论

学校读者我要写书评

暂无评论

On-Chip Traffic Regulation to Reduce Coherence Protocol Cost on a Microthreaded Many-Core Architecture with distributed caches

引用

ACM TRANSACTIONS ON EMBEDDED COMPUTING SYSTEMS 2014年第3-Sup期13卷 103-103页

作者： Yang, Qiang Fu, Jian Poss, Raphael Jesshope, Chris Univ Amsterdam CSA Grp NL-1098 XH Amsterdam Netherlands

When hardware cache coherence scales to many cores on chip, over saturated traffic of the shared memory system may offset the benefit from massive hardware concurrency. In this article, we investigate the cost of a write-update protocol in terms of on-chip memory network traffic and its adverse effects on the system performance based on a multithreaded many-core architecture with distributed caches. We discuss possible software and hardware solutions to alleviate the network pressure. We find that in the context of massive concurrency, by introducing a write-merging buffer with 0.46% area overhead to each core, applications with good locality and concurrency are boosted up by 18.74% in performance on average. Other applications also benefit from this addition and even achieve a throughput increase of 5.93%. In addition, this improvement indicates that higher levels of concurrency per core can be exploited without impacting performance, thus tolerating latency better and giving higher processor efficiencies compared to other solutions.

关键词： Design Experimentation Performance Hardware coherence distributed cache many-core system massive parallelism on-chip memory network write combination

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：