检索结果-内蒙古大学图书馆

Lifelong Machine Learning and root cause analysis for large-scale cancer patient data

JOURNAL OF BIG data 2019年第1期6卷 1页

作者： Pal, Gautam Hong, Xianbin Wang, Zhuo Wu, Hongyi Li, Gangmin Atkinson, Katie Dept Comp Sci Liverpool Merseyside England Xian Jiaotong Liverpool Univ Res Inst Big Data Analyt Suzhou Peoples R China

IntroductionThis paper presents a lifelong learning framework which constantly adapts with changing data patterns over time through incremental learning approach. In many big data systems, iterative re-training high dimensional data from scratch is computationally infeasible since constant data stream ingestion on top of a historical data pool increases the training time exponentially. Therefore, the need arises on how to retain past learning and fast update the model incrementally based on the new data. Also, the current machine learning approaches do the model prediction without providing a comprehensive root cause analysis. To resolve these limitations, our framework lays foundations on an ensemble process between stream data with historical batch data for an incremental lifelong learning (LML) *** descriptionA cancer patient's pathological tests like blood, DNA, urine or tissue analysis provide a unique signature based on the DNA combinations. Our analysis allows personalized and targeted medications and achieves a therapeutic response. Model is evaluated through data from The National Cancer Institute's Genomic data Commons unified data repository. The aim is to prescribe personalized medicine based on the thousands of genotype and phenotype parameters for each *** and evaluationThe model uses a dimension reduction method to reduce training time at an online sliding window setting. We identify the Gleason score as a determining factor for cancer possibility and substantiate our claim through Lilliefors and Kolmogorov-Smirnov test. We present clustering and Random Decision Forest results. The model's prediction accuracy is compared with standard machine learning algorithms for numeric and categorical *** propose an ensemble framework of stream and batch data for incremental lifelong learning. The framework successively applies first streaming clustering technique and then Random Decision Forest Regressor/Classifier to isolat

关键词： Lifelong learning real-time data processing Lambda Architecture Streaming k-means Random Decision Forest Dimension reduction

来源：评论

学校读者我要写书评

暂无评论

real-time data handling and process control--II

Real-time data handling and process control--II

引用

1984年

作者： European Symposium on real-time data Handling and Process Control Kingham E. G. Le Lann G. Malagardis N. E. Commission of the European Communities. Bureau d'orientation de la normalisation en infor

来源：评论

学校读者我要写书评

暂无评论

Internet of Things Architecture for Handling Stream Air Pollution data 2

Internet of Things Architecture for Handling Stream Air Poll...

引用

2nd International Conference on Internet of Things, Big data and Security (IoTBDS)

作者： Kersting, Joschka Geierhos, Michaela Jung, Hanmin Kim, Taehong Univ Paderborn Heinz Nixdorf Inst Furstenallee 11 D-33102 Paderborn Germany Korea Inst Sci & Technol Informat Sci Data Res Ctr Daejeon South Korea

ISBN: (纸本)9789897582455

In this paper, we present an IoT architecture which handles stream sensor data of air pollution. Particle pollution is known as a serious threat to human health. Along with developments in the use of wireless sensors and the IoT, we propose an architecture that flexibly measures and processes stream data collected in real-time by movable and low-cost IoT sensors. Thus, it enables a wide-spread network of wireless sensors that can follow changes in human behavior. Apart from stating reasons for the need of such a development and its requirements, we provide a conceptual design as well as a technological design of such an architecture. The technological design consists of Kaa and Apache Storm which can collect air pollution information in real-time and solve various problems to process data such as missing data and synchronization. This enables us to add a simulation in which we provide issues that might come up when having our architecture in use. Together with these issues, we state reasons for choosing specific modules among candidates. Our architecture combines wireless sensors with the Kaa IoT framework, an Apache Kafka pipeline and an Apache Storm data Stream Management System among others. We even provide open-government data sets that are freely available.

关键词： Wireless Sensor Network Internet of Things Stream data Air Pollution DSMS real-time data processing

来源：评论

学校读者我要写书评

暂无评论

FPGA-based data processing in the Neutron-Sensitive Beam Loss Monitoring System for the ESS Linac 26

FPGA-based Data Processing in the Neutron-Sensitive Beam Los...

引用

26th International Conference on Mixed Design of Integrated Circuits and Systems (MIXDES)

作者： Jablonski, Grzegorz Kittelmann, Irena Dolenc Jalmuzna, Wojciech Kielbik, Rafal Cichalewski, Wojciech Rosengren, Kaj Shea, Thomas Alves, Fabio dos Santos Grishin, Viatcheslav Papaevangelou, Thomas Segui, Laura Zamantzas, Christos Lodz Univ Technol Dept Microelect & Comp Sci Lodz Poland European Spallat Source Lund Sweden Univ Paris Saclay IRFU CEA St Aubin France CERN Geneva Switzerland

ISBN: (纸本)9788363578169

The European Spallation Source (ESS), which is currently under construction, will be a neutron source based on 5 MW, 2 GeV superconducting proton linac. Among other beam instrumentation systems, this high intensity linac requires a Beam Loss Monitoring (BLM) system. An important function of the BLM system is to protect the linac from beam-induced damage by detecting unacceptably high beam loss and promptly inhibiting beam production. In addition to protection functionality, the system is expected to provide the means to monitor the beam losses during all modes of operation with the aim to avoid excessive machine activation. This paper focuses on the FPGA implementation of the real-time data processing in the nBLM system and presents preliminary result of a prototype system installed at LINAC4 at CERN.

关键词： Beam Loss Monitor Accelerator Control System FPGA real-time data processing

来源：评论

学校读者我要写书评

暂无评论

Big data real-time Clickstream data Ingestion Paradigm for E-Commerce Analytics 4

Big Data Real-Time Clickstream Data Ingestion Paradigm for E...

引用

4th International Conference for Convergence in Technology (I2CT)

作者： Pal, Gautam Li, Gangmin Atkinson, Katie Univ Liverpool Dept Comp Sci Liverpool Merseyside England Xian Jiaotong Liverpool Univ Dept Comp Sci Suzhou Peoples R China

ISBN: (纸本)9781538652329

E-Commerce websites track visitors browsing records to uncover their intent and preferences for effective online marketing. User clicks or clickstream is a series of page requests which generates a URL on each click event. Clickstream data is an information trail a user leaves behind while visiting websites. Click URLs can be graphically represented for clickstream reporting. In this work, we study Big data real-time clickstream data ingestion model in e-commerce Domain, which builds on top of standard Big data tools like Kafka, Flume, Spark and Cassandra. In particular, we focus on high velocity, fault-tolerant streaming data acquisition pipelines in a distributed setup rather than mining and searching patterns in it.

关键词： Clickstream real-time data processing Kafka Flume Watermark

来源：评论

学校读者我要写书评

暂无评论

real-time Optimization Algorithm for Intelligent Logistics Transportation Routes Based on Big data Analysis 24

Real-time Optimization Algorithm for Intelligent Logistics T...

引用

International Conference on Machine Intelligence and Digital Applications (MIDA)

作者： Wang, Jinyan Digital Business Acad Heilongjiang Vocat Coll Harbin 150086 Heilongjiang Peoples R China

ISBN: (纸本)9798400718144

In this article, based on big data analysis, the real-time optimization algorithm of intelligent logistics transportation routes is studied for the problems of dataset limitation, lack of real-time performance and single optimization project in traditional logistics transportation routes. This article integrates GPS tracking, traffic monitoring, and weather report data through Talend to remove duplicates, fill in missing values, and handle outliers. time series, traffic flow, and weather feature engineering are constructed to enhance understanding of periodicity and variability. Apache Kafka is utilized to process data in real time and support vector machine algorithms are combined to predict traffic congestion and accidents. Genetic algorithms are used to dynamically generate and optimize transportation routes, and the planning is adjusted through real-time monitoring and response. Finally, the Pareto optimal solution algorithm is used for multi-objective optimization to ensure efficient operation of logistics transportation. In the experiment, the algorithm presented in this article shows significant advantages in transportation time and cost in different cases. In Case 1, the transportation time of the algorithm in this article is only 2.1 hours, and the cost is $45;in the real-time data processing capability test, the response time of the algorithm in this article is the shortest, with 8.7 seconds, 12.1 seconds, and 21.6 seconds, respectively. Meanwhile, the robustness and user satisfaction of the algorithm in this article achieve good results. The algorithm in this article effectively improves the efficiency of logistics transportation and provides reference for similar research.

关键词： Intelligent Logistics Route Optimization Big data Analysis real-time data processing Genetic Algorithm

来源：评论

学校读者我要写书评

暂无评论

Migrate On-Premises real-time data Analytics Jobs Into the Cloud 8

Migrate On-Premises Real-Time Data Analytics Jobs Into the C...

引用

8th IEEE International Conference on data Science and Advanced Analytics (DSAA)

作者： Wu, Huijun Qian, Xiaoyao Crowell, Hulya Pamukcu Singh, Tushar Shulman, Aleks Bhimani, Prashil Maloo, Abhishek Tang, Chunxu Li, Yao Zhang, Lu Ulherr, Chris Twitter Inc San Francisco CA 94103 USA

ISBN: (纸本)9781665420990

Twitter's data platform team is serving a large number of real-time analytics jobs, powering a wide range of data science use cases, from aggregations over time to spam detection. These analytics jobs constitute a crucial step in Twitter's data science infrastructure. As a key part of Twitter's "partly cloudy" strategy, real-time data analytics jobs are being migrated from on-premises into the cloud. We would like to share our migration approach and findings in this paper. The jobs to be migrated vary but follow common patterns, including the "read-modify-write store" and "lambda architecture" patterns. Both patterns can be migrated to the Beam data model in general ways. Besides job patterns, the job IOs are handled by replicating or proxying between on-premises and the cloud. Tests are applied in two phases through monitoring metrics and control tests. A case study demonstrates the business impact of migration. Finally, we discuss lessons learned.

关键词： real-time data processing on-premises cloud migration

来源：评论

学校读者我要写书评

暂无评论

Fast data processing with Spark 2 - Third Edition 3rd

引用

2016年

作者： Sankar, Krishna

ISBN: (数字)9781785882968

ISBN: (纸本)9781785889271;9781785882968

About This BookA quick way to get started with Spark – and reap the rewardsFrom analytics to engineering your big data architecture, we''ve got it coveredBring your Scala and Java knowledge – and put it to work on new and exciting problemsWho This Book Is ForThis book is for developers with little to no knowledge of Spark, but with a background in Scala/Java programming. It''s recommended that you have experience in dealing and working with big data and a strong interest in data *** You Will LearnInstall and set up Spark in your clusterPrototype distributed applications with Spark''s interactive shellPerform data wrangling using the new dataFrame APIsGet to know the different ways to interact with Spark''s distributed representation of data (RDDs)Query Spark with a SQL-like query syntaxSee how Spark works with big dataUnderstand how data scientists and data engineers can use the Spark frameworkImplement machine learning systems with highly scalable algorithmsUse R, the popular statistical language, to work with SparkApply interesting graph algorithms and graph processing with GraphXIn DetailWhen people want a way to process big data at speed, Spark is invariably the solution. With its ease of development (in comparison to the relative complexity of Hadoop), it''s unsurprising that it''s becoming popular with data analysts and engineers *** with the fundamentals, we''ll show you how to get set up with Spark with minimum fuss. You''ll then get to grips with some simple APIs before investigating machine learning and graph processing―throughout we''ll make sure you know exactly how to apply your knowledge. For cloud deployments of Spark, we will look at EC2 (both traditional and EC2MR).You will also learn how to use the Spark shell and load data before finding out how to build and run your own Spark applications. Discover how to manipulate your RDD and get stuck into a range of dataFrame APIs. As if that''s not enough, you''ll also learn some u

关键词： real-time data processing

来源：评论

学校读者我要写书评

暂无评论

MULTIMODAL data COMMUNICATION FOR HUMAN-ROBOT INTERACTIONS

MULTIMODAL DATA COMMUNICATION FOR HUMAN-ROBOT INTERACTIONS

引用

IEEE International Conference on Multimedia and Expo

作者： Wallhoff, Frank Rehrl, Tobias Gast, Juergen Bannat, Alexander Rigoll, Gerhard Tech Univ Munich Dept Elect Engn & Informat Technol Munich Germany

ISBN: (纸本)9781424442904

In this paper. the development of a framework based on the real-time database (RTDB) for processing multimodal data is presented. This framework allows readily integration of input and output modules. Furthermore the asynchronous data streams from different sources can be approximately processed in a synchronous manner. Depending on the included modules, online as well as offline data processing is possible. The idea is to establish a real multimodal interaction system that is able to recognize and react to those situations that are relevant for human-robot interaction.(1)

关键词： multimodal data communication human-robot interaction real-time data processing augmented reality

来源：评论

学校读者我要写书评

暂无评论

The FPGA Pixel Array Detector

引用

NUCLEAR INSTRUMENTS & METHODS IN PHYSICS RESEARCH SECTION A-ACCELERATORS SPECTROMETERS DETECTORS AND ASSOCIATED EQUIPMENT 2013年 701卷 7-16页

作者： Hromalik, Marianne S. Green, Katherine S. Philipp, Hugh T. Tate, Mark W. Gruner, Sol M. SUNY Coll Oswego Dept Comp Sci Oswego NY 13126 USA Cornell Univ Lab Atom & Solid State Phys Ithaca NY 14853 USA Cornell Univ Cornell High Energy Synchrotron Source Ithaca NY 14853 USA

A proposed design for a reconfigurable x-ray Pixel Array Detector (PAD) is described. It operates by integrating a high-end commercial field programmable gate array (FPGA) into a 3-layer device along with a high-resistivity diode detection layer and a custom, application-specific integrated circuit (ASIC) layer. The ASIC layer contains an energy-discriminating photon-counting front end with photon hits streamed directly to the FPGA via a massively parallel, high-speed data connection. FPGA resources can be allocated to perform user defined tasks on the pixel data streams, including the implementation of a direct time autocorrelation function (ACF) with time resolution down to 100 ns. Using the FPGA at the front end to calculate the ACF reduces the required data transfer rate by several orders of magnitude when compared to a fast framing detector. The FPGA-ASIC high-speed interface, as well as the in-FPGA implementation of a real-time ACF for x-ray photon correlation spectroscopy experiments has been designed and simulated. A 16 x 16 pixel prototype of the ASIC has been fabricated and is being tested. (C) 2012 Published by Elsevier B.V.

关键词： X-ray Pixel Array Detectors Field Programmable Gate Array real-time data processing Autocorrelation function

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：