In-memory caching of intermediate data and active combining of data in shuffle buffers have been shown to be very effective in minimizing the recomputation and I/O cost in big data processing systems such as Spark and...
详细信息
In-memory caching of intermediate data and active combining of data in shuffle buffers have been shown to be very effective in minimizing the recomputation and I/O cost in big data processing systems such as Spark and Flink. However, it has also been widely reported that these techniques would create a large amount of long-living data objects in the heap. These generated objects may quickly saturate the garbage collector, especially when handling a large dataset, and hence, limit the scalability of the system. To eliminate this problem, we propose a lifetime-based memory management framework, which, by automatically analyzing the user-defined functions and data types, obtains the expected lifetime of the data objects and then allocates and releases memory space accordingly to minimize the garbage collection overhead. In particular, we present Deca, 1 a concrete implementation of our proposal on top of Spark, which transparently decomposes and groups objects with similar lifetimes into byte arrays and releases their space altogether when their lifetimes come to an end. When systems are processing very large data, Deca also provides field-oriented memory pages to ensure high compression efficiency. Extensive experimental studies using both synthetic and real datasets show that, in comparing to Spark, Deca is able to (1) reduce the garbage collection time by up to 99.9%, (2) reduce the memory consumption by up to 46.6% and the storage space by 23.4%, (3) achieve 1.2x to 22.7x speedup in terms of execution time in cases without data spilling and 16x to 41.6x speedup in cases with data spilling, and (4) provide similar performance compared to domain-specific systems.
A shock tube-based calibration installation for dynamic pressure transducers was designed and manufactured. The hardware of the calibration installation contained a shock tube, two pressure transmitters, three transdu...
详细信息
A shock tube-based calibration installation for dynamic pressure transducers was designed and manufactured. The hardware of the calibration installation contained a shock tube, two pressure transmitters, three transducers for velocity measurement, a temperature transmitter, two data sampling cards and a industrial personal computer, etc. The software of the calibration installation was a data sampling and data processing system. The functions of the software included test conditions monitoring, performance parameters computation, and calibration reports generation, etc. The performance of the calibration installation was tested. The test result showed that the range of the step pressure was 9.60kPa-1.101MPa, the maximum rise time of the step pressure was no more than 1 mu s, the minimum duration of the step pressure plateau was longer than 4ms, the fluctuation of the step pressure plateau was better than +/- 2%, and the maximum 2nd order expand uncertainty of the step pressure was less than 4%. Thus the calibration installation was qualified to be a shock tube-based dynamic pressure standard because that the performance of the calibration installation was verified to meet the requirements of 'Verification Regulation of Dynamic Pressure transducers' and 'Verification Regulation of Dynamic Pressure standards'.
The Information Inference Framework presented in this paper provides a general-purpose suite of tools enabling the definition and execution of flexible and reliable dataprocessing workflows whose nodes offer applicat...
详细信息
The Information Inference Framework presented in this paper provides a general-purpose suite of tools enabling the definition and execution of flexible and reliable dataprocessing workflows whose nodes offer application-specific processing capabilities. The IIF is designed for the purpose of processing big data, and it is implemented on top of Apache Hadoop-related technologies to cope with scalability and high-performance execution requirements. As a proof of concept we will describe how the framework is used to support linking and contextualization services in the context of the OpenAIRE infrastructure for scholarly communication.
The paper considers developed and offered an effective algorithm for solving the block-symmetrical tasks of polynomial computational complexity of dataprocessing modular block-schemes designing. Currently, there are ...
详细信息
The paper considers developed and offered an effective algorithm for solving the block-symmetrical tasks of polynomial computational complexity of dataprocessing modular block-schemes designing. Currently, there are a large number of technologies and tools that allow you to create information systems of any class and purpose. To solve the problems of designing effective information systems, various models and methods are used, in particular, mathematical discrete programming methods. At the same time, it is known that such tasks have exponential computational complexity and can not always be used to solve practical problems. In this regard, there is a need to develop models and methods of the new class, which provide the solution of applied problems of discrete programming, aimed at solving problems of large dimensions. The work has developed and proposed block-symmetric models and methods as a new class of discrete programming problems that allow us to set and solve applied problems from various spheres of human activity. The issues of using the developed models are considered. and methods for computer-aided design of information systems (IS).
This document addresses the various problems of oil control of transport vehicles. The results of observations are presented. The reasons for a significant change in the state of the PBX fleet under the influence of e...
详细信息
This document addresses the various problems of oil control of transport vehicles. The results of observations are presented. The reasons for a significant change in the state of the PBX fleet under the influence of external and internal factors were identified. A block diagram of an automated diagnostic data processing system for oil operating parameters is proposed. Recommendations on maintenance and repair of machines received by the relevant departments of the technical service have been developed. The EPR concept model is developed on the basis of own research.
La présente Norme internationale spécifie les extensions à la distribution du Modèle d'Information Commun (CIM) défini dans la IEC 61970-301. Il définit un jeu standard d'extensio...
详细信息
标准号:
NF C46-909-11-2013
La présente Norme internationale spécifie les extensions à la distribution du Modèle d'Information Commun (CIM) défini dans la IEC 61970-301. Il définit un jeu standard d'extensions du modèle d'information commun (CIM), qui soutient des définitions de messages dans les Parties 3 à 9 de la IEC 61968, la IEC 61968-13 et la IEC 61968-141).Le domaine d'application de ce document est le modèle d'information qui étend la base CIM aux besoins des réseaux de distribution, ainsi qu'à l'intégration dans des systèmes d'information à l'échelle d'entreprise utilisé typiquement dans des compagnies d'électricité. Le modèle d'information est défini en UML, qui est un langage indépendant de la plateforme et programmable informatiquement, qui est utilisé pour créer des messages de définitions de la charge utiles dans différents formats requis. De cette façon, cette norme ne sera pas impactée par la spécification, le développement, et/ou le déploiement des architectures de futures générations, ou bien au travers de l'utilisation de normes ou de moyens propriétaires.
暂无评论