Big Data Todd Goldman March 10, 2017

Are we currently living in the Dark Ages of big data?

 They called it the Dark Ages. The period following the decline of the Western Roman Empire.


Perhaps one day, data experts will look back on today and call this our dark ages. The Dark Ages of Big Data.


Perhaps they will marvel over the obsession many organizations have in hoarding data—especially given their near equal apathy when it comes to actually putting all that data to use.


Over the past few months, travel industry Big Data specialist Mark Ross-Smith has written about some of the excellent ways airlines, hotels and other travel and hospitality related businesses could be converting data,  which is otherwise just sitting around collecting dust, into new high game-changing revenue. He calls it the billion dollar opportunity hidden in plain sight.


He’s not kidding. Scott Kirby, who left American Airlines to become president at United last summer, promptly told analysts that the latter company was losing $1 billion a year in lost revenue due to lackluster handling of data.


It’s mind boggling lethargy in the face of tremendous opportunity, and United is not alone. You can see the same costly neglect throughout every industry. But the good news is more businesses are starting to put their dark data to work in some rather innovative ways. Over time, these innovations will inspire other organizations to come up with innovations of their own. And before we know it, Big Data’s dark age will be no more.


In manufacturing, data is being used to revolutionize the supply chain, where new data clarity is driving efficiencies in partner collaboration, demand forecasting, and delivery networks. In financial services, banks are finally starting to use voice analytics to uncover hidden insights from the everyday conversations providers are having with their customers. Restaurant chains are leveraging their trend analytics in new ways to come up with new recipes and menu offerings that are more likely to draw instant favor from customers.


Some companies are even showing how they’re able to use the data they do have to help inform the data they don’t have. Such is the case with business credit company and Waterline Data customer CreditSafe, which is augmenting weak country data with predictive scores from other countries to improve the quality of its global credit report and scoring models.

Of course, putting dark data to work requires making it discoverable, which no longer has to be a time and money-eating process. 90% of it can be automated for millions in built-in savings from freed manual labor and rationalization of excess databases—not to mention the even bigger new revenue and cost savings opportunities companies stand to gain from dragging their dark data into the light.


Are we nearing the end of Big Data’s dark age?  Yes, and it would be a truly exciting time for all—if not for the many data-deprived organizations that, due to an inability to effectively use their data to compete, may be nearing the end, too.