Big Data Data Processing

What is BIG DATA?

Big Data is also data but with a huge size. Big Data is a term used to describe a collection of data that is huge in volume and yet growing exponentially with time.
In short such data is so large and complex that none of the traditional data management tools are able to store it or process it efficiently.

Examples Of Big Data:

The New York Stock Exchange generates about one terabyte of new trade data per day.

The statistic shows that 500+terabytes of new data get ingested into the databases of social media site Facebook, every day.

This data is mainly generated in terms of photo and video uploads, message exchanges, putting comments etc.

A single Jet engine can generate 10+terabytes of data in 30 minutes of flight time. With many thousand flights per day, generation of
data reaches up to many Petabytes.

Types Of Big Data

BigData’ could be found in three forms:


Any data that can be stored, accessed and processed in the form of fixed format is termed as a ‘structured’ data.
Over the period of time, talent in computer science has achieved greater success in developing techniques for working with such kind of data (where the format is well known in advance) and also deriving value out of it.

However, nowadays, we are foreseeing issues when a size of such data grows to a huge extent, typical sizes are being in the rage of multiple zettabytes.

Looking at these figures one can easily understand why the name Big Data is given and imagine the challenges involved in its storage and processing.

Any data with unknown form or the structure is classified as unstructured data. In addition to the size being huge, un-structured data poses multiple challenges in terms of its processing for deriving value out of it.

A typical example of unstructured data is a heterogeneous data source containing a combination of simple text files, images, videos etc. Now day organizations have wealth of data available with them but unfortunately, they don’t know how to derive value out of it since this data is in its raw form or unstructured format. More details: