4 Reasons Why Your Big Data Analytics Fail
Before we get into the reasons of Big Data Analytics failing, we need to know that, organizations have more data at their disposal than ever before – from terabytes and petabytes of structured data stored in different file formats to large sets of unstructured data being produced from messages, social media posts, shop floor readings and more streaming in each day.
As these mountains of raw information grow, big data analytics is viewed as a promising tool to extract a potential goldmine of valuable business intelligence that is sitting untapped throughout the enterprise. Smart leaders across industries are leading this charge, developing and applying big data analytics strategies to derive actionable insights and fundamentally change the way they make important decisions.
However, industry analyst Gartner predicts that, through 2017, 60 per cent of big data projects will fail to go beyond piloting. Too many organizations lack the required skills, capabilities and culture to truly gain the greatest advantages from their big data.
In my view, there are four primary reasons why organizations fail in their big data analytics initiatives.
1. Data Silos Syndrome
Although a lot of big data is being created, the reality is that most of it tends to get lost in departmental and geographical silos created within large organizations. Many times people aren’t even aware of the existence of the data, let alone have the ability to access it in a timely manner for analysis.
A large part of what makes data accessible and easy to use is the way it’s stored. Given the scale and volume of data that is typically involved in big data analytics, careful consideration needs to be given to the process of data classification and storage to ensure the data can be quickly and easily accessed by multiple stakeholders within the organization. Unless an enterprise finds a way to eradicate the data silos syndrome, it is unlikely to succeed with big data analytics.
A common example of the silos syndrome is when the sales department is unaware of an open customer service request and contacts the customer. The customer naturally gets frustrated, damaging your relationship. A more connected system can notify sales teams when a customer has customer service requests pending.
2. Connectivity Conundrum
Another obstacle that stands in the way of analyzing huge amounts of data is the lack of connectivity between the multiple systems within the organization. In a typical scenario, when the supply chain team needs to make inventory estimates, they must manually collate production and demand estimates using spreadsheets to arrive at inventory norm — a messy and time consuming task. Organizations need to ensure they have the required technology to seamlessly connect between different systems. Unless all the enterprise data is harnessed in a single system, it is difficult to extract meaningful insights. The ability to seamlessly connect to variety of data systems within an organization is a clear business need that must be addressed before implementing big data analytics.
3. Cultural Constraint
As organizations become more data driven, cross functional teams should be able to use the data collected throughout the enterprise and collaborate through empirical decision making. There are very few tools available in the market today to collaboratively analyze datasets. Sometimes there are cultural issues that impede data sharing and collaborative decision making as well. Organizations must often foster a cultural change, in addition to technology change, to succeed in their big data initiatives.
4. End to end enablement
Despite advances in data analytics technology, many enterprises still have multiple components deriving big data insights. One set of tools handles ETL, another handles data visualization and yet another, processing. This means that every big data project involves three different sets of teams, each working with a different set of tools — which all should be in synchronization. Any requirement change may cause delays and increase the time to insight. In projects where decisions depend upon real-time information, any delay undermines the organization’s capability to make correct decisions. There is clearly a business need for one single big data analytics platform that supports the entire analytics life cycle, from analyzing big data to sharing insights.
Organizations that implement big data analytics must take a closer look at these challenges to optimize total cost to value and ensure their big data analytics projects are successful. In my next post, I will recommend the steps organizations should take to avoid these common pitfalls and instead, effectively and successfully tap into their big data and incorporate its insight throughout the enterprise.