The Big data problem is characterized by the so-called 3V features: volume - a huge amount of data, velocity - a high data ingestion rate, and variety - a mix of structured data, semi-structured data, and unstructured...
详细信息
The Big data problem is characterized by the so-called 3V features: volume - a huge amount of data, velocity - a high data ingestion rate, and variety - a mix of structured data, semi-structured data, and unstructured data. The state-of-the-art solutions to the Big data problem are largely based on the MapReduce framework (aka its open source implementation Hadoop). Although Hadoop handles the data volume challenge successfully, it does not deal with the data variety well since the programming interfaces and its associated dataprocessing model are inconvenient and inefficient for handling structured data and graph data. This paper presents epiC, an extensible system to tackle the Big datas data variety challenge. epiC introduces a general Actor-like concurrent programming model, independent of the data processing models, for specifying parallel computations. Users process multi-structured datasets with appropriate epiC extensions, and the implementation of a dataprocessing model best suited for the data type and auxiliary code for mapping that dataprocessing model into epiCs concurrent programming model. Like Hadoop, programs written in this way can be automatically parallelized and the runtime system takes care of fault tolerance and inter-machine communications. We present the design and implementation of epiCs concurrent programming model. We also present two customized data processing models, an optimized MapReduce extension and a relational model, on top of epiC. We show how users can leverage epiC to process heterogeneous data by linking different types of operators together. To improve the performance of complex analytic jobs, epiC supports a partition-based optimization technique where data are streamed between the operators to avoid the high I/O overheads. Experiments demonstrate the effectiveness and efficiency of our proposed epiC.
In recent years, with the increase in environmental awareness, people have become more and more concerned about the effectiveness with which coal burns. Laser-induced breakdown spectroscopy (LIBS) has become an import...
详细信息
In recent years, with the increase in environmental awareness, people have become more and more concerned about the effectiveness with which coal burns. Laser-induced breakdown spectroscopy (LIBS) has become an important way of coal elemental analysis because of its uncomplicated sample handling, remote sensing capability, and superior sensitivity in identifying a wide range of elements, including both major and minor constituents, down to trace levels. However, the complexity of its mechanism of action, the experimental environmental factors, and the presence of matrix effects in its measurement spectrum have affected the measurement accuracy. In this paper, on the basis of introducing the experimental process and principle of LIBS, we summarize and analyze the influence of each factor on the LIBS detection medium, summarize the mainstream model analysis algorithms, and analyze the advantages and disadvantages of each model. While summarizing the LIBS in media detection in recent years, it aims to provide strong support and guidance for subsequent more in-depth exploration and research.
暂无评论