The Price You Pay for Poor Data Quality
When James Lloyd booked a trip from New Zealand to London, he was surprised to discover that the itinerary included a 47 year layover and a return flight that leaves BEFORE his departure. While the system and software was operating smoothly, poor data quality shook up the itinerary. This is a another example of the impact that poor quality of data can have, sometimes with embarrassing impact. The value of a company can be measured by the performance of its data. Luckily the website had a quick and witty social media manager that knew how to address this embarrassing example of poor data quality.
However, data quality often carries serious heavy costs in terms of financial, productivity, missed opportunities, and reputation damage.
Financial Cost of Data Quality
Erroneous decisions made from bad data are not only inconvenient, but also extremely costly. According to Gartner research, “the average financial impact of poor data quality on organizations is $9.7 million per year.”
In additional research for organizations that Gartner has surveyed, the analyst firm “estimate that poor-quality data is costing them on average $14.2 million annually.”
Bad data is bad for business. Ovum Research reported that poor quality data is costing businesses at least 30% of revenues.
Data Quality Cost to Productivity
It goes beyond dollars and cents. Bad data slows employees down, they feel their performance is suffering. According to the Harvard Business Review, the reason bad data costs so much is that decision makers, managers, knowledge workers, data scientists, and others must accommodate it in their everyday work. And doing so is both time-consuming and expensive. The data that is needed may have plenty of errors, and in the face of a critical deadline, many individuals simply make corrections themselves to complete the task at hand.
Data quality truly is a pervasive problem! In fact Forrester reports that “Nearly one third of analysts spend more than 40 percent of their time vetting and validating their analytics data before it can be used for strategic decision-making”
The crux of the problem is that as businesses grow, their business-critical data becomes fragmented. There is no big picture because it’s scattered across applications, including on premise applications. As all this change occurs, business-critical data becomes inconsistent, and no one knows which application has the most up-to-date information. It saps productivity and forces people to do a lot of manual work. The New York Times called this being a janitor. “too much handcrafted work — what data scientists call “data wrangling,” “data munging” and “data janitor work” — is still required. Data scientists, according to interviews and expert estimates, spend from 50 percent to 80 percent of their time mired in this more mundane labor of collecting and preparing unruly digital data, before it can be explored for useful nuggets.”
Data Quality Impacts Reputation
There’s more than just a monetary cost. Poor data quality is an expensive problem that damages reputation. According to the Gartner report, Measuring the Business Value of Data Quality, organizations “make (often erroneous) assumptions about the state of their data and continue to experience inefficiencies, excessive costs, compliance risks and customer satisfaction issues as a result. In effect, data quality in their business goes unmanaged.”
The impact to customer satisfaction undermines a company’s reputation, as customers can take to social media (as in the example at the opening of this article) to share their negative experience. Employees too can start to question the validity of the underlying data when data inconsistencies are left unchecked. This means that they may even ask a customer to validate product, service, and customer data during an interaction — increasing handle times and eroding trust.
Poor Data Quality Can Mean Missed Opportunities
Bad data opens you up to the likelihood of making bad business decisions. Many of those choices can translate directly into missed opportunities. Just as bad data compromises business strategy, it ultimately leads to squandered opportunities downstream as a result of business decisions based on wrong data. Understanding the spending behavior and power of current and potential customers is very important to firms. Many marketers extrapolate this information based on three key categories: current income, modeled net worth, and prior purchasing behavior.
As explained in Data Informed, “Without accurate data on customers, an organization can’t achieve revenue goals. Poor data quality most often affects the ability to reach customers and meet their needs. It harms efforts to maintain accurate customer records, including purchase histories, and thus could lead to missed sales opportunities.”
CASE STUDY: Poor Data Quality at Credit Card Company
Every time a customer swipes their credit card, at any location around the world, that information reaches a central data repository. Before being stored, however, the data is analyzed according to multiple rules, and translated into the company’s unified data format.
With so many transactions, changes can often fly under the radar:
- A specific field is changed by a merchant (e.g. field: “brand name”)
- Field translation prior to reporting fails, and is reported as “null”
- Appears as an erroneous drop in transactions for that merchant’s brand name
- Drop goes unnoticed for weeks, getting lost in the averages of hundreds of other brands they support
Setting back the data analytics effort, the data quality team had to fix the initial data and start analyzing again. In the meantime, the company was pursuing misguided business strategies – costing lost time for all teams, damaging credibility for the data analytics team, adding uncertainty as to the reliability of their data, and creating lost or wrong decisions based on the wrong data.Anodot’s AI-Powered Analytics solution automatically learns the normal behavior for each data stream, flagging any abnormal behavior. Using Anodot, changes leading to issues such as null fields would be immediately alerted on, so that it could be fixed. This prevents wasted time and energy and ensures that decisions are made based on the complete and correct data.
Applying AI-powered Analytics to Ensure Good Data Quality
Reducing the causes of poor data is crucial to stop the negative impact of bad data. An organization’s data quality is ultimately everyone’s business regardless of whether or not they have direct supervision over the data.
Artificial Intelligence can be used to rapidly transform vast volumes of big data into trusted business information. Machine Learning can automatically learn your data metrics’ normal behavior, then discover any anomaly and alert on it. With Anodot’s AI-powered analytics solution, you can rapidly transform your vast volumes of critical data into trusted business information.
Data scientists, business managers, and knowledge workers all have a responsibility to implement the best tools to ensure that false data doesn’t impact critical decisions.