Capitilizing the value of your data – get your basics in place first!

It has been a while since The Economist proclaimed that “data is the new oil” following the tremendous surge in profits of FAAMNG – Facebook, Apple, Amazon, Microsoft, Netflix, Google. Businesses across industries, from utilities to retail, followed suit, hoarding vast amounts of data, strengthening their analytics teams, and seeking use cases that extract value from it. As it turns out, however, this isn’t an easy task, especially for non-typical IT companies.

It does not take long to realize that insights are never better than the underlying data. It’s becoming increasingly clear how crucial it is to have sufficient control over data quality and information governance.

But first things first: before you can improve data quality, you need to understand what it means. Data quality isn’t just a single-dimensional feature. It is a broad term and often described by a number of dimensions, see e.g., 6 dimensions or data quality worksheet:

• completeness – data must be as complete as possible (close to 100%)
• consistency/integrity – there should be no differences in the dataset when comparing two different representations of the same object
• uniqueness – avoiding duplication of data
• timeliness – whether information is available when it is expected and needed
• validity/conformity – data are valid if they conform to the syntax (format, type, range) of their definition
• accuracy – how well the data set represents the real world
• traceability – is it possible to track the data origin and its changes

You will need to work with all of these dimensions. It isn’t enough to improve data completeness if the data does not conform to the expected format.

Moreover, data quality has profound implications for your organization. Although data quality is something the whole organization should focus on, it is natural to focus on teams that actively use the data, depend on its quality, and suffer the most. This usually includes the customer-facing channels, user-facing interfaces, or data warehouse teams that are the first to observe and detect data quality issues. This is often the case when the information governance framework is missing or it is poorly implemented.

Consequently, the information governance framework becomes crucial to ensure sufficient control over data and data quality. Such a framework includes a set of principles, i.e., information governance principles, that the organization establishes and supports, as well as new roles to enforce them. Moreover, organizations often need to establish or strengthen data culture, focus on data quality, and foster the right mindset to ensure quality issues are corrected at the source rather than where they manifest.

The information governance organization itself can operate under a number of different models, i.e.:

• IT driven
  IT takes care of everything, storage, processing, and processes that secure high-quality, structured, and cataloged data
• business driven
  IT only provides the storage infrastructure; the business is in charge of processes to secure data quality
• hybrid model
  IT-driven in some domains, a business driver where it makes the most sense, probably the most pragmatic approach

Improving your information architecture and information governance framework isn’t complicated, but it requires effort and a lot of patience, since it is primarily an organizational and cultural change.
In order to improve the information governance framework and, as a result, improve data quality, you will need to get through at least the following steps:

1. Get an overview of the information architecture and create/improve data models.
   You need to know the current state of the union regarding the most central information entities and how information is modeled, used, and transferred across different parts of your organization.
2. Get an overview of pain points in data quality
   You need to know the specific data-related issues your organization currently faces. Without proper insights, you can’t improve data quality. You need to talk to the business, talk to people around to get enough insights and understanding of the most critical data-related issues they deal with.
3. Create an initial set of governance principles
   Establish the initial governance framework by creating and describing principles for Information Architecture, Enterprise Information Architecture, and data analytics and advanced analytics. Get sufficient backing in the organization.
4. Adjust the organization, create new roles and responsibilities, including roles like information owners, information stewards, data stewards, data scientists, and other roles (see e.g., IBM Redbook, IA governance)
   
5. Finally, consider introducing new tools and technologies to manage information.
   Depending on the results of previous steps and your organization’s needs, you may need to consider new tools to better control your master and reference data. The most obvious one is a Master Data Management system. A Master Data Management system reduces manual work on master data, coordinates master data across systems, keeps it aligned, and detects deviations from the data model.

Although it is tempting to jump in and start implementing new, exciting AI/Machine Learning use cases, the value of this technology depends entirely on underlying data quality and other aspects of information architecture. Data quality and proper information governance are crucial, basic aspects. Without them, the vast amounts of data that you spend lots of effort gathering become not oil, but garbage with little value.