检索结果-内蒙古大学图书馆

14th IEEE/NPSS Real Time Conference on Computing Applications in Nuclear and Plasma Sciences (RT2005)

作者： Barrera, E Ruiz, M López, S Machón, D Vega, J Tech Univ Madrid Elect & Control Syst Dept Madrid Spain Tech Univ Madrid Instrumentat & Appl Acoust Res Grp Madrid Spain EURATOM CIEMAT Assoc Fus Data Acquisit Grp Madrid Spain

ISBN: (纸本)0780391837

This paper describes an architecture model for data acquisition systems based on compact PC! platforms. The aim is to increase real-time data processing capabilities,in experimental environments such as nuclear fusion devices (e.g., ITER). The model has these features: 1) Real-time data acquisition: the system has been provided with real-time capabilities, developing specific data acquisition kernel modules under Linux and RTAI, using COMEDI project drivers;2) Multiprocessor PCI eXtensions for Instrumentation (PXI) Architecture: the model makes possible to add one or more processing cards (in nonsystem slots) to each standard PXI chassis. Several real-time software modules have been developed to allow the communication between the PXI controller and the processing cards. This way the system performance is not restricted to the PXI controller's own performance. This model provides scalability,to the system, adding or removing processing cards;3) Real-time acquired data distribution: with this model it is possible to define how to distribute, in real-time, the data from all acquired signals in the system among the processing cards and the PXI controller;and 4) Dynamic data processing: a software platform has been developed to allow users to dynamically manage their own data processing algorithms in the system. This means that users can start, stop, modify, and replace their data processing algorithms without disrupting the data acquisition process or the rest of the data processing algorithms.

关键词： data acquisition distributed data processing real-time systems thermonuclear fusion

来源：评论

学校读者我要写书评

暂无评论

A distributed Modular data processing Chain Applied to Simulated Satellite Ozone Observations

引用

REMOTE SENSING 2021年第2期13卷 210-210页

作者： Gai, Marco Barbara, Flavio Ceccherini, Simone Cortesi, Ugo Del Bianco, Samuele Tirelli, Cecilia Zoppetti, Nicola Belotti, Claudio Canessa, Bruno Farruggia, Vincenzo Masini, Andrea Keppens, Arno Lambert, Jean-Christopher Arola, Antti Lipponen, Antti Tuinder, Olaf Consiglio Nazl Ric IFAC CNR Ist Fis Appl Nello Carrara I-50019 Sesto Fiorentino Italy Consiglio Nazl Ric INO CNR Ist Nazl Ott I-50019 Sesto Fiorentino Italy Flyby Srl I-57128 Livorno Italy Royal Belgian Inst Space Aeron BIRA IASB B-1180 Brussels Belgium Finnish Meteorol Inst Kuopio 70211 Finland Royal Netherlands Meteorol Inst NL-3731 GA De Bilt Netherlands

Remote sensing of the atmospheric composition from current and future satellites, such as the Sentinel missions of the Copernicus programme, yields an unprecedented amount of data to monitor air quality, ozone, UV radiation and other climate variables. Hence, full exploitation of the growing wealth of information delivered by spaceborne observing systems requires addressing the technological challenges for developing new strategies and tools that are capable to deal with these huge data volumes. The H2020 AURORA (Advanced Ultraviolet Radiation and Ozone Retrieval for Applications) project investigated a novel approach for synergistic use of ozone profile measurements acquired at different frequencies (ultraviolet, visible, thermal infrared) by sensors onboard Geostationary Equatorial Orbit (GEO) and Low Earth Orbit (LEO) satellites in the framework of the Copernicus Sentinel-4 and Sentinel-5 missions. This paper outlines the main features of the technological infrastructure, designed and developed to support the AURORA data processing chain as a distributed data processing and describes in detail the key components of the infrastructure and the software prototype. The latter demonstrates the technical feasibility of the automatic execution of the full processing chain with simulated data. The data processing Chain (DPC) presented in this work thus replicates a processing system that, starting from the operational satellite retrievals, carries out their fusion and results in the assimilation of the fused products. These consist in ozone vertical profiles from which further modules of the chain deliver tropospheric ozone and UV radiation at the Earth's surface. The conclusions highlight the relevance of this novel approach to the synergistic use of operational satellite data and underline that the infrastructure uses general-purpose technologies and is open for applications in different contexts.

关键词： distributed data processing simulated satellite measurements software prototype geo-database

来源：评论

学校读者我要写书评

暂无评论

Prompt: Dynamic data-Partitioning for distributed Micro-batch Stream processing Systems 20

Prompt: Dynamic Data-Partitioning for Distributed Micro-batc...

引用

ACM SIGMOD International Conference on Management of data (SIGMOD)

作者： Abdelhamid, Ahmed S. Mahmood, Ahmed R. Daghistani, Anas Aref, Walid G. Purdue Univ Dept Comp Sci W Lafayette IN 47907 USA

ISBN: (纸本)9781450367356

Advances in real-world applications require high-throughput processing over large data streams. Micro-batching has been proposed to support the needs of these applications. In micro-batching, the processing and batching of the data are interleaved, where the incoming data tuples are first buffered as data blocks, and then are processed collectively using parallel function constructs (e.g., Map-Reduce). The size of a micro-batch is set to guarantee a certain response-time latency that is to conform to the application's service-level agreement. In contrast to tuple-at-a-time data stream processing, micro-batching has the potential to sustain higher data rates. However, existing micro-batch stream processing systems use basic data-partitioning techniques that do not account for data skew and variable data rates. Load-awareness is necessary to maintain performance and to enhance resource utilization. A new data partitioning scheme, termed Prompt is presented that leverages the characteristics of the micro-batch processing model. In the batching phase, a frequency-aware buffering mechanism is introduced that progressively maintains run-time statistics, and provides online key-based sorting as data tuples arrive. Because achieving optimal data partitioning is NP-Hard in this context, a workload-aware greedy algorithm is introduced that partitions the buffered data tuples efficiently for the Map stage. In the processing phase, a load-aware distribution mechanism is presented that balances the size of the input to the Reduce stage without incurring inter-task communication overhead. Moreover, Prompt elastically adapts resource consumption according to workload changes. Experimental results using real and synthetic data sets demonstrate that Prompt is robust against fluctuations in data distribution and arrival rates. Furthermore, Prompt achieves up to 200% improvement in system throughput over state-of-the-art techniques without degradation in latency.

关键词： distributed data processing Micro-batch Stream processing data Partitioning Elastic Stream processing

来源：评论

学校读者我要写书评

暂无评论

Introducing a Novel data Management Approach for distributed Large Scale data processing in Future Computer Clouds

Introducing a Novel Data Management Approach for Distributed...

引用

18th International Conference on Neural Information processing (ICONIP)

作者： Basirat, Amir H. Khan, Asad I. Monash Univ Clayton Sch IT Melbourne Vic 3004 Australia

ISBN: (纸本)9783642249570;9783642249587

Deployment of pattern recognition applications for large-scale data sets is an open issue that needs to be addressed. In this paper, an attempt is made to explore new methods of partitioning and distributing data, that is, resource virtualization in the cloud by fundamentally re-thinking the way in which future data management models will need to be developed on the Internet. The work presented here will incorporate content-addressable memory into Cloud data processing to entail a large number of loosely-coupled parallel operations resulting in vastly improved performance. Using a lightweight associative memory algorithm known as distributed Hierarchical Graph Neuron (DHGN), data retrieval/processing can be modeled as pattern recognition/matching problem, conducted across multiple records and data segments within a single-cycle, utilizing a parallel approach. The proposed model envisions a distributed data management scheme for large-scale data processing and database updating that is capable of providing scalable real-time recognition and processing with high accuracy while being able to maintain low computational cost in its function.

关键词： distributed data processing Neural Network data Mining Associative Computing Cloud Computing

来源：评论

学校读者我要写书评

暂无评论

distributed Human Activity data processing using HASC Tool 11

Distributed Human Activity Data Processing using HASC Tool

引用

13th International Conference on Ubiquitous Computing (UbiComp)

作者： Kawaguchi, Nobuo Ogawa, Nobuhiro Iwasaki, Yohei Kaji, Katsuhiko Nagoya Univ Grad Sch Engn Nagoya Aichi Japan

ISBN: (纸本)9781450309103

To accelerate and simplify human activity recognition research, we have been developing a data processing tool named "HASC Tool." As the activity corpus becomes huge, it is not simple to handle the large number of files because it takes a lot of time to process. In this paper, we propose a distributed data processing mechanism which is implemented in the HASC Tool. By using the system, we can simply scale the local system into distributed processing. We also show the preliminary experimental result.

关键词： Activity Recognition Activity Understandings distributed data processing Large Scale data processing HASC

来源：评论

学校读者我要写书评

暂无评论

A Novel Associative Model of data: Toward a distributed Large-Scale data processing Scheme for Future Computer Clouds

A Novel Associative Model of Data: Toward a Distributed Larg...

引用

11th IEEE International Symposium on Network Computing and Applications (NCA)

作者： Basirat, Amir H. Khan, Asad I. Monash Univ Clayton Sch IT Melbourne Vic 3004 Australia

ISBN: (纸本)9780769547732

Existing cloud frameworks involve isolating low-level operations within an application for data distribution and partitioning. This limits their applicability for many applications with complex data dependency considerations. This paper aims to explore new methods of partitioning and distributing data in the cloud by fundamentally re-thinking the way in which future data management models will need to be developed on the Internet. Loosely-coupled associative computing techniques, which have so far not been considered, can provide the break-through needed for a distributed data management scheme. Using a novel lightweight associative memory algorithm known as Edge Detecting Hierarchical Graph Neuron (EdgeHGN), data retrieval/processing can be modeled as a pattern recognition problem, conducted across multiple records within a single-cycle, utilizing a parallel approach. The proposed model envisions a distributed data management scheme for large-scale data processing and database updating that is capable of providing scalable real-time recognition and processing with high accuracy while being able to maintain low computational cost in its function.

关键词： distributed data processing Neural Network One-shot Learning Associative Computing Cloud Computing

来源：评论

学校读者我要写书评

暂无评论

High performance data processing using distributed computing on the SOLIS project

High performance data processing using distributed computing...

引用

Conference on Advanced Telescope and Instrumentation Control Software II

作者： Wampler, S Natl Solar Observ Tucson AZ 85719 USA

ISBN: (纸本)0819446270

The SOLIS solar telescope collects data at a high rate, resulting in 500 GB of raw data each day. The SOLIS data Handling System (DHS) has been designed to quickly process this data down to 156 GB of reduced data. The DHS design uses pools of distributed reduction processes that are allocated to different observations as needed. A farm of 10 dual-cpu Linux boxes contains the pools of reduction processes. Control is through CORBA and data is stored on a fibre channel storage area network (SAN). Three other Linux boxes are responsible for pulling data from the instruments using SAN-based ringbuffers. Control applications are Java-based while the reduction processes are written in C++. This paper presents the overall design of the SOLIS DHS and provides details on the approach used to control the pooled reduction processes. The various strategies used to manage the high data rates are also covered.

关键词： DHS solar observing distributed data processing Java CORBA

来源：评论

学校读者我要写书评

暂无评论

distributed and Privacy Preserving Graph data Collection in Internet of Thing Systems

引用

IEEE INTERNET OF THINGS JOURNAL 2022年第12期9卷 9301-9309页

作者： Zheng, Xu Tian, Ling Hui, Bei Liu, Xin Univ Elect Sci & Technol China Sch Comp Sci & Engn Chengdu 610000 Peoples R China Trusted Cloud Comp & Big Data Key Lab Sichuan Pro Chengdu 610000 Sichuan Peoples R China Univ Elect Sci & Technol China Sch Informat & Software Engn Chengdu 610054 Peoples R China

Internet of Thing (IoT) systems have been treated as a novel platform for graph data acquisition. Contents like dynamic network topology, organization and control flows, and interactions among monitored objects all contribute to the huge volumes of graph data generated in IoT. These data are believed to brought significant benefits to both the operation and functionalities of IoT systems, especially when combined with cutting-edge Artificial Intelligence techniques. However, these graph data are usually locally collected by data contributors with sensing devices, which could be both partially overlapped as they record same environment, and sensitive as they can indicate private physical status of contributors. Considering all challenges, current solutions for graph data collection in IoT are incapable. Therefore, this article proposes a novel framework for privacy-preserving distributed graph data collection for IoT. The framework allows the graphs kept by data contributors to be partially overlapped, and can help the data broker to efficiently derive the universal view by combining these graphs. The differential privacy is applied for privacy preservation during data collection. The proposed problem aims at minimizing the total bandwidth consumption for graph collection, which is proved to be NP-complete. Then three algorithms are proposed for different circumstances, based on the diverse knowledge and purposes held by the data broker. Finally, both theoretical and numerical analysis have demonstrated the advancement of these methods.

关键词： distributed databases data collection Sensors Internet of Things Monitoring Network topology data models distributed data processing graph data collection Internet of Things (IoT) privacy preservation

来源：评论

学校读者我要写书评

暂无评论

An Approach to the distributed Generation of Event Logs Based on data from Heterogeneous Monitoring Devices 9

An Approach to the Distributed Generation of Event Logs Base...

引用

9th Mediterranean Conference on Embedded Computing (MECO)

作者： Bekeneva, Yana A. St Petersburg Electrotech Univ LETI Dept Comp Sci & Engn St Petersburg Russia

ISBN: (数字)9781728169491

ISBN: (纸本)9781728169491

Modern monitoring systems are often consist of various heterogeneous devices generating data in different format with different attributes. To analyze such data using process mining techniques, it is necessary to define the most important features of events. One of the widely used data format for process mining *** format that allows usage of four attributes only. The paper presents an approach for dynamic determination of set of attributes and features of the events to be processed. The approach presented is tested on real world data set describing vehicle movement on the territory of the factory.

关键词： event logs distributed data processing monitoring systems monitoring devices

来源：评论

学校读者我要写书评

暂无评论

distributed Sentiment Analysis for Geo-Tagged Twitter data 30

Distributed Sentiment Analysis for Geo-Tagged Twitter Data

引用

30th IEEE Signal processing and Communications Applications Conference (SIU)

作者： Zengin, Muhammed Said Arslan, Rabia Akgun, Mehmet Burak TOBB Ekon & Teknol Univ Bilgisayar Muhendisligi Bolumu Ankara Turkey

ISBN: (纸本)9781665450928

The ever-increasing frequency of sharing on social media makes these platforms one of the primary sources of data for computational social science studies. Similarly, examining and analyzing large scale social media data-sets is crucial for governments as well as companies. However, as the amount of data increases, insights that need to be derived from the data using artificial intelligence based models becomes more and more demanding in terms of processing power. In fact, hardware requirements might dramatically increase if the insights are needed under real-time or near-real time constraints. In this study, we developed a distributed sentiment analysis model that utilizes a large social media data-set. 16 million tweets have been collected and grouped by the originating city. The sentiment analysis model was produced by fine-tuning the pre-trained BERT model. distributed big data analytics engine, Apache Spark, is used to execute the trained model in a distributed fashion. For evaluation purposes, the prediction time on a single compute unit is compared with the distributed prediction time. Sentiment analysis model has been executed separately for each of the data-groups corresponding to 81 provinces. The data-set containing 16 million tweets used in this study, the Turkish sentiment analysis model produced, the distributed prediction code developed for Apache Spark and all the results of the study can be accessed from the address https://***/.

关键词： Big data distributed data processing sentiment analysis BERT

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：