A good part of this decade had been ruled by Hadoop with its different software suits as far as big data was concerned. The Hadoop distributed file system or HDFS had revolutionized data storage all by itself. The concept of utilizing data to for various avails was quite old.
The technology required to store and process significantly large amounts of data was not available at that point. When big data did come into industrial practice, it was confined within the elite sphere of enterprises owing to the sheer lack of affordability.
HDFS changed the scenario for the small and midsized enterprises which wanted capitalize on the data. Hence, it was pretty big for a better part of the decade. The data processing unit of Hadoop, Mapreduce was never as popular as the HDFS but still a lot of people used as their primary tool.
The decline of Hadoop
We can divide the journey of some software in a few steps which can help you imagine the graph of its rise, stagnancy and decline. The once the software is introduced there is a peak of expectation; then there is usually a form of disillusionment that the software is not capable of actually creating the value that was expected from it.
Here we can see the curve drop. After this there is a period of stagnation, a plateau of productivity. In this phase there is parity between the expectations and the result. People start investing with caution. Then comes the inevitable decline prompted by the arrival of new technologies and changes in the market.
Hadoop’s position is arguably in between the second and the third stage. Some people would say that Hadoop is as good as dead. But I would say there is still value to be acquired from a Hadoop certification, more so in Southeast Asia.
The real situation
With the advent and popularity of cloud the distributed file system has taken the back seat, and quite naturally. With the elasticity and simple handling that cloud provides, distributed storage does really seem a back dated option.
Hence most of the organizations are getting their date out of HDFS and those who are still using it will probably not continue with it for very long. Does this mean that Hadoop does not have any value? Nothing could be further away from truth. As part of Hadoops compute unit Apache Spark which works in tandem with Hadoop is one of the best known names in the data analytics sphere.
Spark is famous for its processing speed and with the elasticity of data storage provided by cloud, Spark is killing it in the market.
So, if you are planning to learn Hadoop, go for it by all means, because there are a handful of people left in the planet who are really skilled in Hadoop. But make sure that you are learning stuff that is actually in tune with the industry requirements.