10 common pitfalls in data management

August 28, 2018 5:17 AM

10 common pitfalls that threaten data quality strategies

Business Team Investment Entrepreneur Trading Concept

“Implementing a data quality strategy is not as simple as installing a tool or a one-time fix,” explains Patty Haines, president and founder of Chimney Rock Information Solutions, Inc., a consultancy that aids organizations in building business intelligence and analytics environments by providing data warehouse and business intelligence services, solutions and mentoring. Writing in the report "Ten mistakes to avoid when implementing a data quality strategy," from TDWI, Haines says “Organizations across the enterprise need to work together to identify, assess, remediate, and monitor data with the goal of continual data improvements.” Haines offers her advice on 10 top challenges to a successful data quality strategy.

Mistake 1: Assuming your enterprise data is clean and accurate

“No matter how many safeguards are built into your enterprise applications, data can still be entered and managed inaccurately,” Haines says. “The business will continue to change and grow. Data entry teams will be given new responsibilities. As part of the data quality strategy, applications will be enhanced, business processes will be adjusted, and training must be provided to ensure data is entered and managed accurately.”

Mistake 2: Assuming your enterprise data has only one business definition

“If differences in the definition and use of data continue, it can allow poor quality data to be entered, managed and reported,” Haines says. “The data quality strategy must include the business community, data governance, and subject matter experts working together to determine consistent and agreed-upon definitions to improve the quality of data.”

Mistake 3: Skipping the assessment phase

“The best approach is to start with completing an assessment of your organization’s applications and data,” Haines advises. “Business people, subject matter experts and data governance teams work together to first identify and rank the critical business domains, along with data elements deemed critical to each domain. The critical data elements of each business domain are profiled and analyzed to determine their quality. Metrics are developed to provide a high-level view of the data quality for each business domain and associated critical data elements.”

Mistake 4: Not profiling and interrogating data values

MISTAKE-FOUR-NOT-PROFILING-AND-INTERROGATING-DATA-VALUES.jpg

“Profiling and evaluating data is a first step for the business and data governance teams to better understand what their data actually looks like, how it compares to other data values, and how to determine the quality of data,” Haines says.

Mistake 5: Not creating and using data quality standards

MISTAKE-FIVE-NOT-CREATING-AND-USING-DATA-QUALITY-STANDARDS.jpg

“The more consistent and standardized data evaluation can be, the better data quality within each application will be,” Haines stresses. “In addition, the data quality strategy will be easier to build and manage when it is based on data categories that are being monitored and reported in a standard and consistent manner.”

Mistake 6: Not including templates and standard processes as part of the data quality strategy

“Standard data quality reports and metrics also need to be developed and shared with the business and data governance teams,” according to Haines. “This will help them understand the need for a data quality strategy and why the data quality tasks are required. Many times, the business community has been sheltered from poor-quality data through improved user interfaces. Showing them examples of actual enterprise data will educate them about what they will look for when evaluating and analyzing data.”

Mistake 7: Not following the data quality road map

MISTAKE-SEVEN-NOT-FOLLOWING-THE-DATA-QUALITY-ROAD-MAP.jpg

“The data quality road map is developed with input from support team members, database developers, the business community, and the data governance team to ensure a solid sequence of projects is defined,” Haines says. “The road map brings together sets of domains that make business and technical sense. Consideration is given to the size, technology, stability of the applications, and availability of the right team members to be part of the data quality projects.”

Mistake 8: Building the data quality strategy in one large project

MISTAKE-EIGHT-BUILDING-THE-DATA-QUALITY-STRATEGY-IN-ONE-LARGE-PROJECT.jpg

“For the initial data strategy project, start with a business domain that has a high chance of success, involves fewer organizational groups, and can be completed in a short amount of time,” Haines notes. “This project should have a clear set of success criteria that is regularly evaluated and monitored. Smaller projects afford you the opportunity to test ideas in a smaller environment to ensure they perform as expected.”

Mistake 9: Viewing technology as the entire solution

MISTAKE-NINE-VIEWING-TECHNOLOGY-AS-THE-ENTIRE-SOLUTION.jpg

“Though it’s true that technology continues to move forward and software vendors provide better and faster tools with each new release, data quality management is a three-legged stool with data governance, business processes, and technology each providing a leg,” Haines says.

Mistake 10: Not continually monitoring and evaluating data

MISTAKE-TEN-NOT-CONTINUALLY-MONITORING-AND-EVALUATING-DATA.jpg

“A data quality strategy is not a one-time data clean-up event,” Haines stresses. “It requires metrics to provide insight concerning the value and usability of data assets over time. Developing these metrics is mainly the task of the business and data governance teams. They will develop data quality metrics to show data quality, data quality scoring methods, and measurement processes, both currently and over time. The goal is that monitoring and reporting these metrics will show improved data as the data quality enhancements are implemented.”