Sumit Sharma

Disruptions to how all data will be structured and processed

leave a comment »

I’m no expert in databases but I’d like to talk about an issue that is quitely sliping into the background, though is very important and crucial to our enjoyment of that quick presentation of search results on our favourite search engine, or why we can easily search through the plethora of posts on our Facebook page….

This whole new approach to contextualizing this massive amount of data will lead to disruptions in data storage and processing, as evidenced by the popularity of the “no SQL” movement today. Google’s MapReduce and Apache’s Hadoop are two popular frameworks for distributed computing and mass data storage and processing that are manifestations of this “no SQL” movement today. Yahoo, Google and Facebook amongst other data powerhouses are increasingly using such frameworks to organize and process queries that span their immense repositories of data in a manner that is more efficient and time consuming than if they were to use traditional relational databases to store data and process information and queries from. This is not to say that this is the end of relational databases, because it is still possible extract efficiencies by intelligent indexing and optimizing how data is stored and processed, but Hadoop and MapReduce are definitely disruptive forces to be taken very seriously.


Written by Sumit

December 6, 2009 at 7:38 am

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

%d bloggers like this: