检索结果-内蒙古大学图书馆

real-time data processing for serial crystallography experiments

IUCRJ 2025年第1期12卷 97-108页

作者： White, Thomas Schoof, Tim Yakubov, Sergey Tolstikova, Aleksandra Middendorf, Philipp Karnevskiy, Mikhail Mariani, Valerio Henkel, Alessandra Klopprogge, Bjarne Hannappel, Juergen Oberthuer, Dominik Aquino, Ivan De Gennaro Egorov, Dmitry Munke, Anna Sprenger, Janina Pompidor, Guillaume Taberman, Helena Gruzinov, Andrey Meyer, Jan Hakanpaeae, Johanna Gasthuber, Martin Deutsch Elektronen Synchrotron DESY Hamburg Germany Deutsch Elektronen Synchrotron DESY Ctr Data & Comp Nat Sci CDCS Hamburg Germany Deutsch Elektronen Synchrotron DESY Ctr Free Electron Laser Sci CFEL Hamburg Germany SLAC Natl Accelerator Lab Linac Coherent Light Source Menlo Pk CA USA Hamburg Ctr Ultrafast Imaging Hamburg Germany Oak Ridge Natl Lab ORNL Oak Ridge TN USA

We report the use of streaming data interfaces to perform fully online data processing for serial crystallography experiments, without storing intermediate data on disk. The system produces Bragg reflection intensity measurements suitable for scaling and merging, with a latency of less than 1 s per frame. Our system uses the CrystFEL software in combination with the ASAP::O data framework. In a series of user experiments at PETRA III, frames from a 16 megapixel Dectris EIGER2 X detector were searched for peaks, indexed and integrated at the maximum full-frame readout speed of 133 frames per second. The computational resources required depend on various factors, most significantly the fraction of non-blank frames ('hits'). The average single-thread processing time per frame was 242 ms for blank frames and 455 ms for hits, meaning that a single 96-core computing node was sufficient to keep up with the data, with ample headroom for unexpected throughput reductions. Further significant improvements are expected, for example by binning pixel intensities together to reduce the pixel count. We discuss the implications of real-time data processing on the 'data deluge' problem from recent and future photon-science experiments, in particular on calibration requirements, computing access patterns and the need for the preservation of raw data.

关键词： macromolecular crystallography serial crystallography X-ray crystallography real-time data processing

来源：评论

学校读者我要写书评

暂无评论

The real-time data processing framework for blockchain and edge computing

引用

ALEXANDRIA ENGINEERING JOURNAL 2025年 120卷 50-61页

作者： Gao, Zhaolong Yan, Wei Shandong Technol & Business Univ Sch Comp Sci & Technol Yantai 264005 Peoples R China Shandong Technol & Business Univ Coll Innovat & Entrepreneurship Coll Blockchain Applicat Technol Yantai 264005 Peoples R China

The rapid growth of IoT has increased the demand for large-scale data processing. However, traditional centralized methods struggle with real-time requirements and data security. This paper introduces VCDTSNet, a novel real-time IoT data processing framework that combines blockchain and edge computing. By integrating deep learning models like VGG, ConvLSTM, and DNN, VCD-TSNet effectively performs spatial feature extraction, temporal modeling, and decision-making, while using blockchain to ensure data integrity and privacy. Experimental results demonstrate that VCD-TSNet outperforms baseline models in classification accuracy, prediction precision, and real-time performance. For instance, on the BoT-IoT dataset, the classification accuracy reaches 97.5%, throughput increases to 920 TPS, and response time stays below 85 ms. This study validates the model's effectiveness and highlights its potential in large-scale IoT environments, offering efficient, secure solutions for real-time data processing. It also provides insights for future improvements in frameworks that combine edge computing with blockchain.

关键词： Blockchain real-time data processing IoT Edge computing Deep learning

来源：评论

学校读者我要写书评

暂无评论

real-time data processing Pipeline for Trigger Readout Board-Based data Acquisition Systems

引用

IEEE TRANSACTIONS ON NUCLEAR SCIENCE 2022年第7期69卷 1765-1772页

作者： Malige, A. Korcyl, G. Firlej, M. Fiutowski, T. Idzik, M. Korzeniak, B. Lalik, R. Misiak, A. Molenda, A. Moron, J. Rathod, N. Salabura, P. Smyrski, J. Swientek, K. Wintz, P. Jagiellonian Univ Fac Phys Astron & Appl Comp Sci Marian Smoluchowski Inst Nucl Phys PL-30348 Krakow Poland Akad Gorniczo Hutniczej Wydzial Fizyki & Informatyki Stosowanej PL-30059 Krakow Poland Forschungszentrum Nucl Phys Inst D-52425 Julich Germany

Large-scale physics experiments running at high interaction rates place a high demand on the data acquisition system (DAQ) responsible for transporting the data from the detector to the storage. The antiProton ANihilation at DArmstadt (PANDA) at the facility for anti-proton and ion research (FAIR) is one such experiment of the future that will not use fixed hardware triggers;instead, the event selection is based on real-time feature extraction, filtering, and high-level correlations. A firmware framework for such real-time data processing has been developed and tested with hardware setup for a PANDA Forward Tracker (FT) prototype. The solution is applicable for other detector subsystems based on the so-called Trigger Readout Board (TRB) data read-out system.

关键词： Detectors Pipelines Feature extraction Hardware data acquisition Anodes real-time systems data acquisition systems (DAQs) real-time data processing tracking detectors

来源：评论

学校读者我要写书评

暂无评论

real-time data processing for In-line Monitoring of a Pharmaceutical Coating Process by Optical Coherence Tomography

Real-time Data Processing for In-line Monitoring of a Pharma...

引用

Conference on Biophotonics - Photonic Solutions for Better Health Care IV

作者： Markl, Daniel Ziegler, Jakob Hannesschlaeger, Guenther Sacher, Stephan Buchsbaum, Andreas Leitner, Michael Khinast, Johannes G. Res Ctr Pharmaceut Engn GmbH Inffeldgasse 13-2 A-8010 Graz Austria Res Ctr Non Destruct Testing GmbH A-4040 Linz Austria Graz Univ Technol Inst Proc & Particle Engn A-8010 Graz Austria

ISBN: (纸本)9781628410778

Coating of tablets is a widely applied unit operation in the pharmaceutical industry. Thickness and uniformity of the coating layer are crucial for efficacy as well as for compliance. Not only due to different initiatives it is thus essential to monitor and control the coating process in-line. Optical coherence tomography (OCT) was already shown in previous works to be a suitable candidate for in-line monitoring of coating processes. However, to utilize the full potential of the OCT technology an automatic evaluation of the OCT measurements is essential. The automatic evaluation is currently implemented in MATLAB and includes several steps: (1) extraction of features of each A-scan, (2) classification of A-scan measurements based on their features, (3) detection of interfaces (air/coating and coating/tablet core), (4) correction of distortions due to the curvature of the bi-convex tablets and the oblique orientation of the tablets, and (5) determining the coating thickness. The algorithm is tested on OCT data acquired by moving the sensor head of the OCT system across a static tablet bed. The coating thickness variations of single tablets (i.e., intra-tablet coating variability) can additionally be analysed as OCT allows the measurement of the coating thickness on multiple displaced positions on one single tablet. Specifically, the information about those parameters emphasises the high capability of the OCT technology to improve process understanding and to assure a high product quality.

关键词： Optical coherence tomography pharmaceutical coating process in-line monitoring real-time data processing

来源：评论

学校读者我要写书评

暂无评论

real-time data processing on graphics processors

Real-time data processing on graphics processors

引用

Conference on Photonics, Applications in Astronomy, Communications, Industry, and High-Energy Physics Experiments IV

作者： Lipowski, J Warsaw Univ Technol Inst Elect Syst PL-00665 Warsaw Poland

ISBN: (纸本)081946211X

The paper concerns possible usage of graphics processing units (GPU) for general real-time data processing. It presents main differences between classic CPU architectures and so called stream architectures realized by modem GPUs. All the considerations revolve around effective processing of large data sets such as multidimensional signals. The paper is accompanied by a application which uses GPU to process 2D image data with a user defined convolution filter.

关键词： GPU stream architectures parallel processing real-time data processing OpenGL

来源：评论

学校读者我要写书评

暂无评论

real-time data processing in supply chain management: revealing the uncertainty dilemma

引用

INTERNATIONAL JOURNAL OF PHYSICAL DISTRIBUTION & LOGISTICS MANAGEMENT 2019年第10期49卷 1003-1019页

作者： Lechler, Sabrina Canzaniello, Angelo Rossmann, Bernhard von der Gracht, Heiko A. Hartmann, Evi Friedrich Alexander Univ Erlangen Nuremberg Chair Supply Chain Management Nurnberg Germany Steinbeis Univ Berlin Sch Int Business & Entrepreneurship Futures Studi Herrenberg Germany Friedrich Alexander Univ Erlangen Nuremberg Nurnberg Germany

Purpose Particularly in volatile, uncertain, complex and ambiguous (VUCA) business conditions, staff in supply chain management (SCM) look to real-time (RT) data processing to reduce uncertainties. However, based on the premise that data processing can be perfectly mastered, such expectations do not reflect reality. The purpose of this paper is to investigate whether RT data processing reduces SCM uncertainties under real-world conditions. Design/methodology/approach Aiming to facilitate communication on the research question, a Delphi expert survey was conducted to identify challenges of RT data processing in SCM operations and to assess whether it does influence the reduction of SCM uncertainty. In total, 14 prospective statements concerning RT data processing in SCM operations were developed and evaluated by 68 SCM and data-science experts. Findings RT data processing was found to have an ambivalent influence on the reduction of SCM complexity and associated uncertainty. Analysis of the data collected from the study participants revealed a new type of uncertainty related to SCM data itself. Originality/value This paper discusses the challenges of gathering relevant, timely and accurate data sets in VUCA environments and creates awareness of the relationship between data-related uncertainty and SCM uncertainty. Thus, it provides valuable insights for practitioners and the basis for further research on this subject.

关键词： Uncertainty Supply chain management VUCA Delphi survey real-time data processing

来源：评论

学校读者我要写书评

暂无评论

real-time data processing With Lambda Architecture

Real-Time Data Processing With Lambda Architecture

引用

作者： Omkar Ashok Malusare San Jose State University

学位级别：硕士

data has evolved immensely in recent years, in type, volume and velocity. There are several frameworks to handle the big data applications. The project focuses on the Lambda Architecture proposed by Marz and its application to obtain real-time data processing. The architecture is a solution that unites the benefits of the batch and stream processing techniques. data can be historically processed with high precision and involved algorithms without loss of short-term information, alerts and insights. Lambda Architecture has an ability to serve a wide range of use cases and workloads that withstands hardware and human mistakes. The layered architecture enhances loose coupling and flexibility in the system. This a huge benefit that allows understanding the trade-offs and application of various tools and technologies across the layers. There has been an advancement in the approach of building the LA due to improvements in the underlying tools. The project demonstrates a simplified architecture for the LA that is maintainable.

关键词： Lambda Architecture (LA) Batch processing Stream processing real-time data processing Systems Architecture

来源：评论

学校读者我要写书评

暂无评论

Map Matching Algorithm for real-time data processing of Non-route GPS data in Seoul

引用

KSCE JOURNAL OF CIVIL ENGINEERING 2021年第9期25卷 3511-3522页

作者： Han, Yohee Kim, Youngchan Ku, Jisun Jung, Yeonghun Roh, Jeongrae Univ Seoul Dept Transportat Engn Seoul 02504 South Korea Seoul Metropolitan Govt Transportat Informat Div Seoul 03153 South Korea Zetalux Syst Co Ltd Solut Div Seoul 05713 South Korea

Global Positioning System (GPS) data are satellite-based longitude and latitude coordinates for which map matching with road links, which are electronic maps, may be used in traffic operation management. Various studies have been conducted to improve the accuracy of map matching;however, research on the application of map matching algorithms for real-time data processing systems has not been widely conducted. Therefore, this study investigated a map matching algorithm that can shorten the processing time required for application to systems that process large volumes of GPS data collected for a wide network. To reduce the processing time, a candidate link selection method and a grid-type geospatial indexing process were designed. The processing time was analyzed based on an emulator before applying the map matching algorithm to the Transport Operation and Information Service Center, which generates the real-time travel speed data in Seoul City. The results obtained from comparing the prior and new algorithms showed a 96% decrease in the processing time. We believe that the findings of this study will benefit the real-time processing of GPS data, which are expected to be used in various ways in traffic operations and management in the future.

关键词： Map matching real-time data processing GPS big data processing time Geospatial indexing

来源：评论

学校读者我要写书评

暂无评论

Forecast Model Update Based on a real-time data processing Lambda Architecture for Estimating Partial Discharges in Hydrogenerator

引用

SENSORS 2020年第24期20卷 7242页

作者： Pereira, Fabio Henrique Bezerra, Francisco Elanio Oliva, Diego Martha de Souza, Gilberto Francisco Chabu, Ivan Eduardo Santos, Josemir Coelho Nagao Junior, Shigueru Nabeta, Silvio Ikuyo Nove de Julho Univ UNINOVE Informat & Knowledge Management Grad Program BR-01525000 Sao Paulo Brazil Nove de Julho Univ UNINOVE Ind Engn Grad Program BR-01525000 Sao Paulo Brazil Univ Sao Paulo EPUSP Polytech Sch BR-05508010 Sao Paulo Brazil

The prediction of partial discharges in hydrogenerators depends on data collected by sensors and prediction models based on artificial intelligence. However, forecasting models are trained with a set of historical data that is not automatically updated due to the high cost to collect sensors' data and insufficient real-time data analysis. This article proposes a method to update the forecasting model, aiming to improve its accuracy. The method is based on a distributed data platform with the lambda architecture, which combines real-time and batch processing techniques. The results show that the proposed system enables real-time updates to be made to the forecasting model, allowing partial discharge forecasts to be improved with each update with increasing accuracy.

关键词： autoregressive forecasting model lambda architecture partial discharges power hydrogenerators real-time data processing

来源：评论

学校读者我要写书评

暂无评论

FPGA based real-time data processing DAQ system for the Mercury Imaging X-ray Spectrometer

FPGA based real-time data processing DAQ system for the Merc...

引用

16th Euromicro Conference on Digital System Design (DSD)

作者： Aschauer, Florian Stechele, Walter Treis, Johannes Max Planck Inst Extraterr Phys Giessenbachstr D-85741 Garching Germany Tech Univ Munich Lehrstuhl Integrierte Syst D-80333 Munich Germany PNDetector GmbH D-81739 Munich Germany

ISBN: (纸本)9780769550749

This paper presents a new DAQ system with real-time data processing capability for X-ray detector systems. The data processing algorithms are working directly on the serialised input data stream from the detector system and run in real-time on a FPGA processing system. The DAQ system includes new algorithms for dynamic adaptation of important processing parameters during a measurement. These dynamic algorithms enable better measurement precisions, autonomous system operation and avoid detector recalibration cycles. The developed and presented data processing algorithms are optimised for low memory bandwidth and hardware resource utilisation. High data reduction and live monitoring of important system parameters are achieved with the real-time processing. The concept of the new DAQ system was tested and evaluated with the Mercury Imaging X-ray Spectrometer (MIXS) [5], [2]. MIXS is an instrument on board of the ESA corner stone mission BepiColombo [1] which will be launched 2015 into an orbit around Mercury. The 64x64 detector matrix used in the MIXS [3], [4] detector system for X-ray detection delivers 6000 frames per second and provides an optimal evaluation platform for the new DAQ system.

关键词： FPGA DAQ real-time data processing X-ray MIXS DEPFET

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：