My job as a Business Intelligence Practice Leader is to help companies develop a reliable process for making data-driven decisions. BI projects usually begin with a road mapping phase during which our team 1) meets with stakeholders to work out goals for the project, 2) assesses the company’s current data-tracking capabilities, and 3) lays out a plan for how we’ll take them from where they are now to where they’d like to be. Over the years, I’ve recognized a recurring theme at the outset of most of the BI projects I’ve worked on—to put it bluntly, a lot of the data businesses rely on to make decisions is really crappy.
Solutions to problems with data quality are seldom solely technological, especially when the problems don’t trace back to any software or hardware—they trace back to people. While there are some fantastic tools to help companies get a grip on their data issues, throwing all the technology in the world at data quality problems without resolving the underlying foundational concerns is simply a waste of time and money. That’s where the idea of Data Governance comes into play. Data Governance takes not only the techy part into account, but the people and process parts as well. To give you an idea how this works, I’ll cover two examples of how poor data quality can negatively impact businesses, one that is hypothetical, and one from a real project I worked on. Afterward, I’ll talk about two Data Governance software platforms designed to address these types of problems
One of the areas where Data Governance is emerging as a really important topic is healthcare. Large health networks often grow through acquisitions; they may buy a doctor’s office, then a clinic, then a hospital. The likelihood that these previously independent establishments are using the same practice management software is pretty slim. So, if I go into a doctor’s office and end up being sent to a hospital for some tests, my information will probably be entered into two separate systems in two different formats. This makes my case very difficult to track because no one has ready access any holistic view of what’s going on in these different offices—even though they’re all part of the same health network. Now what if I also had to go to a third, or even a fourth place? You can see how messy this can get.
3 Departments, 3 Different Terms
I was once getting started on a BI project and had to acquaint myself with the client company’s data-entry procedures. After having a representative from one department take me through some basic tasks, I left that office and went into the one right next door. Right away, I was shocked to discover that the representative in this office was using the term “invoice” to label the exact same information that I’d heard described as a “closed order” in the other office. Things only got more confusing in the third office I visited, where the term was “shipped sale.” So, each of the three offices—in the same hallway!—had its own name for the exact same thing. Obviously, anyone trying to make decisions based on data associated with one term or the other is either going to miss some important information or run into a whole lot of confusion. I can’t imagine what a meeting involving all 3 departments at the same time would be like!
What is Data Governance?
Data governance is basically quality control for your business’s information. It provides a holistic view of your data, no matter what industry your business is in. The old saying, “Garbage in, garbage out” captures the idea of what we’re aiming to avoid. Data Governance encompasses the business rules that go into generating data and includes things like standard definitions and entry procedures, safeguards against duplication and extraneous noise, and protocols for managing the paths that data takes from entry to consumption to ensure that you have a solid basis for making informed decisions about the future direction of your company.
But again Data Governance isn’t just about technology. Data Stewards are business users (not necessarily IT people) who help their companies agree on the procedures and terms they want to standardize. In addition to managing data storage platforms, Data Stewards also bring accountability to the system by keeping track of the standards and making sure that everyone in the company stays on the same page and follows the same rules. They also serve as champions or “chief flag wavers” for the Data Governance project—which is one of the most important parts of the job.
Microsoft’s Master Data Services
With the more recent versions of SQL Server, Master Data Services (MDS) is available for Data Stewards. This platform provides a robust user interface over a repository to house all of your important organization-level information, storing it in one place to make it easier to arrive at a holistic view of what’s going on with your business and to ensure that any data used in one application is the same data used in another. MDS also comes equipped with tools that allow the Data Steward to clean away extraneous noise, correct any duplications or redundancies, create hierarchies, and ensure the application of standard business rules. The end result is that the Steward can sign off on the data that is ultimately presented to decision-makers in the company, thus assuring them that they’re basing their decisions on quality information.
Microsoft’s Data Quality Services
Data Quality Services (DQS) helps Data Stewards ensure compliance with the standard procedures. For instance, a Steward may decide that name entries should always include both first and last names concatenated together. DQS automatically applies this rule to any information that goes into the master data repository. If a user in the company does type in just a first name, there are several options available for how to deal with the incomplete information. An automatic message could be sent to the person who typed it in. Or it could be sent to a temporary holding area which generates a report every day on all the instances of when people failed to adhere to the standard procedures. However it’s handled, though, the important thing is that there’s accountability so that over time everyone learns the same rules and uses them as part of the normal routine to generate meaningful data that is applicable company-wide.
In my experience, poor data quality due to a lack of good governance is one of the most common reasons business projects fail. But improving the quality of the information you have on-hand for your business is just the first step. What often happens is that once business leaders start getting a better idea of what’s going with their company they discover several opportunities for streamlining their practices they never would’ve even considered before. That’s why your Business Intelligence solution should be thought of as an ongoing process of constantly refining the way you make decisions for the future of your business.