Challenges and Techniques for Testing of Big Data

作     者：Naveen Garg Sanjay Singla Surender Jangra 

作者机构：Research Scholar Department of Computer Science Engineering I.K. Gujral Punjab Technical University Jalandhar India Assistant Professor Amity University Uttar Pradesh India Professor & HoD Department of Computer Science Engineering IET Bhaddal India Assistant Professor Department of IT Engineering HCTM Kaithal India 

出 版 物：《Procedia Computer Science》 

年 卷 期：2016年第85卷

页      面：940-948页

主　　题：Big Data Testing Verasity Hadoop 

摘      要：Big Data, the new buzz word in the industry, is data that exceeds the processing and analytic capacity of conventional database systems within the time necessary to make them useful. With multiple data stores in abundant formats, billions of rows of data with hundreds of millions of data combinations and the urgent need of making best possible decisions, the challenge is big and the solution bigger, Big Data. Comes with it, new advances in computing technology together with its high performance analytics for simpler and faster processing of only relevant data to enable timely and accurate insights using data mining and predictive analytics, text mining, forecasting and optimization on complex data to continuously drive innovation and make the best possible decisions. While Big Data provides solutions to complex business problems like analyzing larger volumes of data than was previously possible to drive more precise answers, analyzing data in motion to capture opportunities that were previously lost, it poses bigger challenges in testing these scenarios. Testing such highly volatile data, which is more often than not unstructured generated from myriad sources such as web logs, radio frequency Id (RFID), sensors embedded in devices, GPS systems etc. and mostly clustered data for its accuracy, high availability, security requires specialization. One of the most challenging things for a tester is to keep pace with changing dynamics of the industry. While on most aspects of testing, the tester need not know the technical details behind the scene however this is where testing Big Data Technology is so different. A tester not only needs to be strong on testing fundamentals but also has to be equally aware of minute details in the architecture of the database designs to analyze several performance bottlenecks and other issues. Like in the example quoted above on In-Memory databases, a tester would need to know how the operating systems allocate and de-allocate memory and u