Too much CRAP – Big Data is here to stay

All the data.
All the time.

That pretty much sums up The Age of Big Data in which we now find ourselves. Organizations that learn how to make use of the data will prosper.
There are datastreams being generated by all the ways we are connected to the internet.

This data is being collected and stored, although there is too much of it to organize, so it’s CRAP (create, replicate, append, process) as Charles Fan calls it; “generated by machines, coming in large quantities at high velocity.”

The solution? Easy. Make the data accessible in a way that gives it structure. That’s where we come in.
 

So the great challenge is to use this data. If the data is not used then, there is little point in storing it, especially at cost.

Q: What should (big) data be used for?
A: For making decisions.

In other words, why bother to save all that data, unless to put it to good use, and that means helping organizations make decisions. Those decisions can be based directly on human behavior, for example to decide what content to display to a viewer, based on Social Media preferences, or based on machine signals.

Automated signals, such as all the purchases made by supermarket customers with bonus cards, which can help managers decide which products to stock & display, create datasets too large for humans to manage.

That’s where solutions such as Opentracker come in. You don’t need a big engineering team to get value out of the data if you use a suitable data collection and reporting system.

In our case, we will:

• provide the engineering
• host the data
• provide a simple interface
• allow the end-user to send signals (write)
• and ask the data questions (read)

Click here for an article about understanding Big Data.

Because there are new and massive databases coming into existence, new technologies are needed in order to manage the data. The point of efficient data management is that it takes processing power to make calculations. Therefore, the new & efficient way which has evolved is to leave the data as it is; CRAP, and sort it out only on a need-to-know-basis.

That’s where Opentracker comes in. What we have built is a distributed-data technology that allows users to slice & dice at will.

In other words, you send the data, it’s stored, but not structured. The only thing that is structured is the tool (an API: application program interface) that allows you to ask the data questions.

Please get in touch if you have any questions.

PS- And why is Big Data not going away? Because with ever-increasing amounts of data, the likelihood of the data being organized in any meaningful way becomes increaingly smaller.