The Future Of The Database

Twiddling thumbs between sessions at a database conference, Mayank Sharma ponders on the impact of the web on databases…

- Mayank Sharma

This is the information era, where almost everything is data. The scale at which this data is being collected and used is growing at an exponential rate. The speed at which organisations are ingesting, storing and processing data is hard to fathom; it’s very common these days for even small-scale companies to process terabytes and even petabytes of data. In order for this data to be crunched into meaningful information, it first needs to be housed in a database.

Databases have become a part of your daily routine in more ways than you can imagine. These days you don’t even have to be on a computer or use your smartphone to interact with a database. For instance, when you purchase items at the local supermarket, chances are there’s an inventory database that automatically updates itself while you checkout. The same thing happens when you borrow a book from the library, withdraw cash from the ATM or buy a movie ticket – the list is virtually endless. In fact, it’s safe to say that a majority of your daily chores will involve some sort of interaction with a database.

In addition to these traditional uses of a database that are fulfilled by what is known as a relational database, the internet has had a great influence on databases in both form and function. The explosion of social media platforms such as Twitter, Facebook, Instagram and the like have ushered in a new generation of databases that are designed to overcome the scalability limitations of the earlier varieties. These hold vast quantities of different kinds of data that just cannot be stored and processed, at least not efficiently, by traditional relational database systems.

Taking a look at the evolution of the mechanisms for data storage, retrieval and processing will help us better appreciate the challenges of an often unappreciated and unglamorous branch of computer science.