Lessons learnt from over a decade of Hadoop

Back in 2006 Apache Hadoop emerged and soon began to revolutionize the nascent world of big Data. It’s one of the key factors that helped shape a new industry and -- with the cloud -- helped drive a raft of new consumer industries and business services. But the data lakes of Hadoop became a challenge to manage, and many Big Data and analytical projects became more of a quagmire than a sparkling lake of truth. What’s more the number compute engines overpopulated. They were operationally complex and demanded specialized skills to maintain. Those data lakes became very disparate compute engines,… [Continue Reading]