检索结果-内蒙古大学图书馆

System Status Aware Hadoop Scheduling Methods for Job Performance Improvement

IEICE TRANSACTIONS ON INFORMATION AND SYSTEMS 2015年第7期E98D卷 1275-1285页

作者： Kawarasaki, Masatoshi Watanabe, Hyuma Univ Tsukuba Tsukuba Ibaraki 3058550 Japan

MapReduce and its open software implementation Hadoop are now widely deployed for big data analysis. As MapReduce runs over a cluster of massive machines, data transfer often becomes a bottleneck in job processing. In this paper, we explore the influence of data transfer to job processing performance and analyze the mechanism of job performance deterioration caused by data transfer oriented congestion at disk I/O and/or network I/O. Based on this analysis, we update Hadoop's Heartbeat messages to contain the real time system status for each machine, like disk I/O and link usage rate. This enhancement makes Hadoop's scheduler be aware of each machine's workload and make more accurate decision of scheduling. The experiment has been done to evaluate the effectiveness of enhanced scheduling methods and discussions are provided to compare the several proposed scheduling policies.

关键词： Hadoop MapReduce distributed computing task scheduling job performance

来源：评论

学校读者我要写书评

暂无评论

A Secure Data Infrastructure for Personal Manufacturing Based on a Novel Key-Less, Byte-Less Encryption Method

引用

IEEE ACCESS 2020年 8卷 40039-40056页

作者： Vedeshin, Anton Dogru, John Mehmet Ulgar Liiv, Innar Ben Yahia, Sadok Draheim, Dirk 3D Control Syst Inc San Francisco CA 94129 USA Tallinn Univ Technol Dept Software Sci EE-12618 Tallinn Estonia Univ Oxford Ctr Technol & Global Affairs Oxford OX1 3UQ England Univ Tunis El Manar Fac Sci Tunis Tunis 2092 Tunisia Tallinn Univ Technol Informat Syst Grp EE-12618 Tallinn Estonia

We are witnessing the advent of personal manufacturing, where home users and small and medium enterprises manufacture products locally, at the point and time of need. The impressively fast adoption of these technologies indicates this approach to manufacturing can become a key enabler of the real-time economy of the future. In this paper, we contribute a secure and dependable infrastructure and architecture for that new paradigm. Our solution leverages physical limitations of the computational process into a defense strategy that makes distributed file storage and transfer highly secure. The main idea is to replace asymmetric or public-key encryption functions with an unkeyed, collision, second preimage, and preimage resistant cryptographic hash function. Such a cryptosystem does not have an inverse function . We challenge each block hash against the full hash table to recreate the original message. To illustrate the approach, we describe secured protocols that provide a number of desirable properties during both data storage and streaming. Similar to proof-of-work blockchain consensus algorithms, we parameterized the solution based on the amount of infrastructure available. Experiments show the proposed method can recalculate hashes for a 3-dimensional of 256(3) at an average of 14 revisions per second, and one revision every 5 minutes for a bigger matrix of 4096(3). The increase in cloud infrastructure cost is insignificant compared to the level of protection offered.

关键词： Communication system security computer aided manufacturing content distribution networks data security data storage systems distributed computing information security intelligent manufacturing systems technology social factors virtual manufacturing

来源：评论

学校读者我要写书评

暂无评论

Guest Editorial Special Issue on Blockchain and Economic Knowledge Automation

引用

IEEE TRANSACTIONS ON SYSTEMS MAN CYBERNETICS-SYSTEMS 2020年第1期50卷 2-8页

作者： Yuan, Yong Wang, Shouyang Olson, David L. Lambert, James H. Wang, Fei-Yue Rong, Chunming Stavrou, Angelos Zhang, Jun Tang, Qiang Baldimtsi, Foteini Yang, Laurence T. Wu, Desheng Chinese Acad Sci Inst Automat Beijing 100190 Peoples R China Univ Stavanger Ctr IP Based Serv Innovat N-4036 Stavanger Norway George Mason Univ Dept Comp Sci Fairfax VA 22030 USA Wuhan Univ Sch Elect Engn & Automat Wuhan 430072 Peoples R China New Jersey Inst Technol Dept Comp Sci Newark NJ 07102 USA St Francis Xavier Univ Dept Comp Sci Antigonish NS B2G 2W5 Canada Univ Chinese Acad Sci Sch Econ & Management Beijing 100190 Peoples R China Chinese Acad Sci Acad Math & Syst Sci Beijing 100190 Peoples R China Univ Nebraska Coll Business Adm Lincoln NE 68588 USA Univ Virginia Dept Syst & Informat Engn Charlottesville VA 22903 USA

Blockchain, as an emerging decentralized architecture and distributed computing paradigm underlying Bitcoin and other cryptocurrencies, has attracted intensive attention in both research and applications recently. Blockchain, especially powered by chain-coded smart contracts, has the full potential of revolutionizing increasingly centralized cyber-physical-social systems (CPSSs) for constructions and applications, and reshaping traditional knowledge automation workflows. The key advantage of blockchain technology lies in the fact that it can enable the establishment of secured, trusted, and decentralized autonomous ecosystems for various scenarios, especially for better usage of the legacy devices, infrastructure, and resources.

关键词： Special issues and sections Blockchain Cyrptocurrency Smart contracts Cryptography distributed computing Computer architecture Cyber-physical systems

来源：评论

学校读者我要写书评

暂无评论

RPYFMM: Parallel adaptive fast multipole method for Rotne-Prager-Yamakawa tensor in biomolecular hydrodynamics simulations

引用

COMPUTER PHYSICS COMMUNICATIONS 2018年 227卷 99-108页

作者： Guan, W. Cheng, X. Huang, J. Huber, G. Li, W. McCammon, J. A. Zhang, B. Univ N Carolina Dept Math Chapel Hill NC 27599 USA Ohio State Univ Coll Pharm Med Chem & Pharmacognosy Columbus OH 43210 USA Shandong Univ Technol Sch Transportat & Vehicle Engn Zibo Shandong Peoples R China Univ Calif San Diego Dept Chem & Biochem La Jolla CA 92093 USA Univ Calif San Diego Dept Pharmacol La Jolla CA 92093 USA Indiana Univ Sch Informat Comp & Engn Ctr Res Extreme Scale Technol Bloomington IN 47404 USA

RPYFMM is a software package for the efficient evaluation of the potential field governed by the Rotne-Prager-Yamakawa (RPY) tensor interactions in biomolecular hydrodynamics simulations. In our algorithm, the RPY tensor is decomposed as a linear combination of four Laplace interactions, each of which is evaluated using the adaptive fast multipole method (FMM) (Greengard and Rokhlin, 1997) where the exponential expansions are applied to diagonalize the multipole-to-local translation operators. RPYFMM offers a unified execution on both shared and distributed memory computers by leveraging the DASHMM library (DeBuhr et al., 2016, 2018). Preliminary numerical results show that the interactions for a molecular system of 15 million particles (beads) can be computed within one second on a Cray XC30 cluster using 12,288 cores, while achieving approximately 54% strong-scaling efficiency. Program summary Program Title: RPYFMM: Parallel Adaptive FMM for RPY Tensor Program Files doi: http://***/10.17632/zpbjvy8whp.1 Licensing provisions: BSD 3-clause Programming language: C++ Nature of problem: Evaluate the Rotne-Prager-Yamakawa tensor matrix-vector multiplications describing the hydrodynamics interaction in biomolecular systems. Solution method: The Rotne-Prager-Yamakawa tensor is decomposed as a linear combination of four Laplace interactions, each of which is evaluated using the new version of adaptive fast multipole method [1]. Additional Comments: RPYFMM is built on top of the DASHMM library and the Asynchronous Multi-Tasking HPX-5 runtime system. DASHMM is automatically downloaded during installation and HPX-5 is available at http://***/. (C) 2018 Elsevier B.V. All rights reserved.

关键词： Brownian dynamics Rotne-Prager-Yamakawa tensor Hydrodynamics interactions Fast multipole method DASHMM distributed computing

来源：评论

学校读者我要写书评

暂无评论

IMPLEMENTATION OF SIMULTANEOUS ATTENUATION AND DETECTOR RESPONSE CORRECTION IN SPECT

引用

IEEE TRANSACTIONS ON NUCLEAR SCIENCE 1988年第1期35卷 778-783页

作者： TSUI, BMW HU, HB GILLAND, DR GULLBERG, GT UNIV UTAH MED CTRDEPT RADIOLSALT LAKE CITYUT 84132 UNIV N CAROLINA CURRICULUM BIOMED ENGNCHAPEL HILLNC 27514

Simultaneous correction of nonuniform attenuation and detector response was implemented in single-photon-emission computed tomography (SPECT) image reconstruction. A ray-driven projector-backprojector that exactly models attenuation in the reconstructed image slice and the spatially variant detector response was developed and used in the iterative maximum-likelihood algorithm for the correction. A computer-generated heart-lung phantom was used in simulation studies to compare the simultaneous correction method with an intrinsic attenuation correction method using a smoothing filter, and intrinsic attenuation correction method using a deconvolution filter, and a modified Chang attenuation correction method using a nonuniform attenuation distribution. The results demonstrate that the present method provides more-accurate quantitation and superior image quality.< >

关键词： Attenuation Detectors Image reconstruction Filters Computed tomography Maximum likelihood detection Iterative algorithms distributed computing Imaging phantoms Computational modeling

来源：评论

学校读者我要写书评

暂无评论

Parallel matrix factorization for recommender systems

引用

KNOWLEDGE AND INFORMATION SYSTEMS 2014年第3期41卷 793-819页

作者： Yu, Hsiang-Fu Hsieh, Cho-Jui Si, Si Dhillon, Inderjit S. Univ Texas Austin Dept Comp Sci Austin TX 78712 USA

Matrix factorization, when the matrix has missing values, has become one of the leading techniques for recommender systems. To handle web-scale datasets with millions of users and billions of ratings, scalability becomes an important issue. Alternating least squares (ALS) and stochastic gradient descent (SGD) are two popular approaches to compute matrix factorization, and there has been a recent flurry of activity to parallelize these algorithms. However, due to the cubic time complexity in the target rank, ALS is not scalable to large-scale datasets. On the other hand, SGD conducts efficient updates but usually suffers from slow convergence that is sensitive to the parameters. Coordinate descent, a classical optimization approach, has been used for many other large-scale problems, but its application to matrix factorization for recommender systems has not been thoroughly explored. In this paper, we show that coordinate descent-based methods have a more efficient update rule compared to ALS and have faster and more stable convergence than SGD. We study different update sequences and propose the CCD++ algorithm, which updates rank-one factors one by one. In addition, CCD++ can be easily parallelized on both multi-core and distributed systems. We empirically show that CCD++ is much faster than ALS and SGD in both settings. As an example, with a synthetic dataset containing 14.6 billion ratings, on a distributed memory cluster with 64 processors, to deliver the desired test RMSE, CCD++ is 49 times faster than SGD and 20 times faster than ALS. When the number of processors is increased to 256, CCD++ takes only 16 s and is still 40 times faster than SGD and 20 times faster than ALS.

关键词： Recommender systems Missing value estimation Matrix factorization Low rank approximation Parallelization distributed computing

来源：评论

学校读者我要写书评

暂无评论

STREETS OF BYZANTIUM - NETWORK ARCHITECTURES FOR FAST RELIABLE BROADCASTS

引用

IEEE TRANSACTIONS ON SOFTWARE ENGINEERING 1985年第6期11卷 546-554页

作者： BABAOGLU, O DRUMMOND, R Department of Computer Science Cornell University

A site broadcasting its local value to all other sites in a fault-prone environment is a basic paradigm in the development of reliable distributed systems. Time complexity lower limits and network connectivity requirements for reliable broadcast protocols in point-to-point communication networks are well known. Here, examination focuses on the reliable broadcast problem in distributed systems with broadcast networks as the fundamental communication architecture. It is demonstrated how properties of such network architectures can be employed to effectively restrict the externally visible behavior of faulty processors. These methods are used to derive simple protocols that implement reliable broadcast in only 2 rounds, independent of the failure upper bounds.

关键词： Network Partitions Byzantine Agreement distributed computing Ethernet Fault Tolerance

来源：评论

学校读者我要写书评

暂无评论

Multi-functional secure data aggregation schemes for WSNs

引用

AD HOC NETWORKS 2018年 69卷 86-99页

作者： Zhang, Ping Wang, Jianxin Guo, Kehua Wu, Fan Min, Geyong Cent S Univ Sch Informat Sci & Engn Changsha Hunan Peoples R China Shanghai Jiao Tong Univ Dept Comp Sci & Engn Shanghai Peoples R China Univ Exeter Coll Engn Math & Phys Sci Dept Math & Comp Sci Exeter Devon England

Secure data aggregation schemes are widely adopted in wireless sensor networks, not only to minimize the energy and bandwidth consumption, but also to enhance the security. Statistics obtained from data aggregation schemes often fall into three categories, i.e., distributive, algebraic, and holistic. In practice, a wide range of reasonable aggregation queries are combinations of several different statistics. Providing multi-functional aggregation support is also a primary demand for data preprocessing in data mining. However, most existing secure aggregation schemes only focus on a single type of statistics. Some statistics, especially holistic ones (e.g., median), are often difficult to compute efficiently in a distributed mode even without considering the security issue. In this paper, we first propose a new Multi-functiOnal secure Data Aggregation scheme (MODA), which encodes raw data into well-defined vectors to provide value-preservation, order-preservation and context-preservation, and thus offering the building blocks for multi-functional aggregation. A homomorphic encryption scheme is adopted to enable in-ciphertext aggregation and end-to-end security. Then, two enhanced and complementary schemes are proposed based on MODA, namely, RandOm selected encryption based Data Aggregation (RODA) and COmpression based Data Aggregation (CODA). RODA can significantly reduce the communication cost at the expense of slightly lower but acceptable security on a leaf node, while CODA can dramatically reduce communication cost with the lower aggregation accuracy. The performance results obtained from theoretic analysis and experimental evaluation of three real datasets under different scenarios, demonstrate that our schemes can achieve the performance superior to the most closely related work. (C) 2017 Elsevier B.V. All rights reserved.

关键词： Wireless sensor networks Secure data aggregation distributed computing

来源：评论

学校读者我要写书评

暂无评论

Leader-Based Consensus of Heterogeneous Nonlinear Multiagent Systems

引用

MATHEMATICAL PROBLEMS IN ENGINEERING 2014年第1期2014卷 1-6页

作者： Sun, Tairen Pan, Yongping Yu, Haoyong Jiangsu Univ Sch Elect & Informat Engn Zhenjiang 212013 Peoples R China S China Univ Technol Minist Educ Key Lab Autonomous Syst & Networked Control Guangzhou 510640 Guangdong Peoples R China Natl Univ Singapore Dept Biomed Engn Singapore 117575 Singapore

This paper considers the leader-based consensus of heterogeneous multiple agents with nonlinear uncertain systems. Based on the information obtained from the following agents' neighbors, leader observers are designed by the following agents to estimate the leader's states and nonlinear dynamics. Then, to achieve leader-based consensus, adaptive distributed controllers are designed for the following agents to track the designed corresponding leader observers. The effectiveness of the leader observers and distributed consensus controllers are illustrated by formal proof and simulation

关键词： NONLINEAR systems MULTIAGENT systems UNCERTAINTY (Information theory) COMPUTER simulation distributed computing

来源：评论

学校读者我要写书评

暂无评论

An object-oriented programming framework for the parallel dynamic analysis of structures

引用

COMPUTERS & STRUCTURES 2002年第1期80卷 77-84页

作者： Modak, S Sotelino, ED Purdue Univ Sch Civil Engn W Lafayette IN 47907 USA Comp & Struct Inc Berkeley CA 94704 USA

This work addresses the time history analysis of structures subjected to dynamic loads using high performance computing environments. Structural mechanics, parallel computing, and object-oriented programming methodologies are integrated to design and implement frameworks for parallel and sequential transient finite element analysis (TFE++ and PTFE++). The object-oriented approach is employed to facilitate extensibility, reusability, maintainability and simplicity of the resulting software. Parallel processing concepts and algorithms are used in the design of PTFE++. An application has been developed to demonstrate the developed frameworks. It has been found that PTFE++ provides an efficient way to analyze large structural systems. (C) 2002 Elsevier Science Ltd. All rights reserved.

关键词： parallel processing distributed computing object-oriented programming structural dynamics transient finite element analysis time-stepping algorithms

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：