Enterprise Data World Webinar: A Strategic Approach to Data Quality

A Strategic Approach to Data Quality: A Dataversity Webinar

Laura Sebastian-Coleman, Ph.D., IQCP

October 15, 2013

Maybe if we plan for

better data, we’ll

actually produce

better data…

Confidential property of Optum. Do not distribute or reproduce without express permission from Optum. 2

Goals & Agenda

• Goals

– Get you to think strategically about data quality improvement

– Provide concrete suggestions about actions you can take to assess your

organization’s readiness to adopt a strategic approach

• Agenda

– Brief introduction

– An important disclaimer

– Review definitions and some assumptions and about how following concepts

are connected: Strategy, Data, Data Quality, Data Quality Assessment, Data

Quality Strategy

– Review the 12 Directives of Data Quality Strategy

– Discuss the actions you can take to assess your organization’s readiness to

adopt a strategic approach to improving data quality

– Comments, questions, discussion


About Optum

• Optum is a leading information and technology-enabled health services

business dedicated to helping make the health system work better for

everyone.

• With more than 35,000 people worldwide, Optum delivers intelligent,

integrated solutions that modernize the health system and help to

improve overall population health.

• Optum solutions and services are used at nearly every point in the

health care system, from provider selection to diagnosis and treatment,

and from network management, administration and payments to the

innovation of better medications, therapies and procedures.

• Optum clients and partners include those who promote wellness, treat

patients, pay for care, conduct research and develop, manage and

deliver medications.

• With them, Optum is helping to improve the delivery, quality and cost

effectiveness of health care.


About me

• 10+ years experience in data quality in the health care industry.

• Working on the IT side of things, in data warehousing

• My thinking about data quality and data governance has been influenced by the demands of data warehousing

• Author of Measuring Data Quality for Ongoing Improvement : A Data Quality Assessment Framework (2013).

• Have also worked in banking, manufacturing, distribution, commercial insurance, and academia.

• This combination of experiences has influenced my understanding of data, quality, and measurement.

• It has also led me to conclude that most organizations do not think strategically about data.

– Data is still treated more as a by-product of processes than as a product itself.

– Most organizations assume data from different processes should fit together, and they feel pain when that assumption turns out to be incorrect.


Disclaimer!

• None of the 12 Directives is new. And none of them should be surprising.

• They are synthesized from thought leaders in Data Quality and Data

Governance – and from thought leaders in product quality who preceded them.

Many of them will be familiar and, I hope, obvious. All are interconnected.

• What is new is a set of concrete steps you can take to assess your

organization’s readiness to move forward with an appropriate sub-set of the

directives.

• The directives are also based on the assumption that most of us face at least

these common challenges:

– Organizational systems have evolved over time

through a series of undocumented compromises.

– Data is created in disparate places within the organization

(silos) and it does not always fit together in expected ways.

– Critical knowledge about data is stored in the heads of

Subject Matter Experts and is not directly accessible to

data consumers.


Strategy

• Military origin of the word: strategia , the Greek word for generalship,

was first used to describe military operations and movement.

– Strategy is planning for a set of engagements (battles and other military

interventions) that will achieve the overall goal of a war.

– Tactics describe how each of these engagements will be carried out.

– Tactical “success” that does not contribute to strategy—“winning the battle

but losing the war”—is not success at all.

• What does this tell us about any strategy?

– Strategy is intentional:

– To be strategic is to plan for success by thinking in terms of time and space.

– What do you want to accomplish?

– Where do you want to be within a defined time period?

– How do you get there?


Strategy

• The purpose of a business or an organizational strategy is to align

work efforts with long-term goals and to plan for achieving overall

goals.

• Strategy requires:

– A clear vision & mission—where the organization wants to go

– An understanding of current state—where it is now

– Tactics—how it will move from where it is to where it wants to be

• Strategy provides criteria to set priorities and to make decisions when

conflicting needs arise between or within teams.

– Of course, for strategy to be effective, people must actually use these

criteria.


Conceptual Relation between Strategic Planning,

Tactical Execution, and Strategic Success


Warning!

• Strategy is exciting and motivating!

• People like to know where they are going and how to get there. Sometimes they even try to do everything at once.

• Unfortunately, that does not usually work.

• Strategy is about planning. So, to repeat, a plan requires defining:

– Where the organization wants to go

– Where it is now

– How it will move from where it is to where it wants to be

• Data Strategy requires defining these goals in relation to the role that data plays within your organization.


Data

• The New Oxford American Dictionary (NOAD) defines data as “facts and statistics collected together for reference or analysis.”

• ASQ defines data as “A set of collected facts.” ASQ identifies two kinds of numerical data: “measured or variable data … and counted or attribute data.”

• ISO defines data as “re-interpretable representation of information in a formalized manner suitable for communication, interpretation, or processing” (ISO 11179).

• I define data as: abstract representations of selected characteristics of real-world objects, events, and concepts, expressed and understood through explicitly definable conventions related to their meaning, collection, and storage.

• Observations:

– Data tries to tell the truth about the world (“facts”)

– Data is formal – it has a shape

– Data is created through human choices, so to understand data’s “truth” you need to understand the choices that influence its shape

– That is, you need to understand how data effects – brings about – its representation of the world

– Today, data almost always means electronically stored data


Data Quality

• Data Quality / Quality of Data:

– The level of quality of data represents the degree to which data meets the expectations of data consumers, based on their intended use of the data.

– Because data also serves a semiotic function (it serves as a sign of something other than itself), data quality is also directly related to the perception of how well data effects (brings about) this representation.

• Observations:

– High-quality data meets expectations for use and for representational effectiveness to a greater degree than low-quality data.

– Assessing the quality of data requires understanding those expectations and determining the degree to which the data meets them.

– To understand the quality of data, you need to understand where it comes from and how it works

• The concepts the data represents

• The processes that created data

• The systems through which the data is created

• The known and potential uses of the data


Assessment

• Assessment is the process of evaluating or estimating the nature,

ability, or quality of a thing.

• Like measurement, assessment requires comparison.

• But importantly! Assessment implies drawing a conclusion about—evaluating—the

object of the assessment, whereas measurement does not always imply doing so.

• Assessing organizational readiness for a strategic approach to data quality

means assessing your organization’s culture:

• Taking a hard, objective look at how people work together, how they manage change,

resolve conflict, reward success, etc.

• Making general observations about what works well and what does not work well in

your organization can help you identify obstacles and opportunities to improving data

quality.

• Quantifying characteristics that help you explain changes that need to take place.


What is a Data Strategy?

• The concept of strategy can be applied to many facets of an organization: Financial strategy, HR strategy, Product Strategy, Technology strategy, etc.

• “Strategies” for different parts of an organization need to be aligned. At the very least, they should not contradict each other, but ideally they should support each other.

• Strategy is future-oriented, but strategic success depends on knowing your starting point: Assessing current state in order to remove obstacles to success and create the conditions for success.

• Translated to Data Strategy, this means

– Recognizing how data supports your organization’s overall mission

– Applying techniques from other areas of quality improvement to data (e.g., problem definition, process analysis, measurement, root cause analysis)

• So that …..

– Actions can be defined to create an organizational commitment to producing high quality data.


Strategic Alignment

• Data Strategy – an organization’s plan for improving the quality of data

it produces and uses, in order to meet current and future organizational

goals

• Should align with the organization’s overall mission and its

– Infrastructure strategy – a plan for building ensuring the organization’s

technology infrastructure is sound and positioned to meet future needs.

– Technology strategy – a plan for ensuring that investment in technology is

coordinated across parts of the organization, that systems can “talk” to each

other.

– Data Governance strategy – the organization’s approach to establishing

decision rights and accountabilities for information-related processes*

– Data Stewardship strategy – its plan for ensuring ongoing engagement of

business and technology staff in the care and management of organizational

data assets.

* Gwen Thomas, The Data Governance Institute


Without a Strategy… Any road will take you where?

Alice came to a fork in the road.

Alice: “Would you tell me, please, which way I ought to go from here?”

The Cheshire Cat: “That depends a good deal on where you want to get to.”

Alice: “I don't much care where.”

The Cheshire Cat: “Then it doesn't much matter which way you go.”

Alice: “So long as I get somewhere.

The Cheshire Cat: “Oh, you're sure to do that, if only you walk long enough.”


The 12 Directives

• Component pieces of a strategic approach to data quality that can be applied within any organization – they support each other.

• Not a process – these are not “steps” to data quality strategy. They are ways of orienting your organization toward demanding higher quality data.

• Where you start depends on your organization & your role within it. So we will discuss assessing organizational readiness to move forward in these areas: – Directive 1: Obtain Management Commitment to Data Quality

– Directive 2: Treat Data as an Asset

– Directive 3: Apply Resources to Focus on Quality

– Directive 4: Build Explicit Knowledge of Data

– Directive 5: Treat Data as a Product of Processes which can be Measured and Improved

– Directive 6: Recognize Quality is Defined by Data Consumers

– Directive 7: Address the Root Causes of Data Problems

– Directive 8: Measure Data Quality, Monitor Critical Data

– Directive 9: Hold Data Producers Accountable for the Quality their Data

– Directive 10: Provide Data Consumers with the Knowledge the Require for Data Use

– Directive 11: Data will continue to evolve – Plan for evolution

– Directive 12: Data Quality goes Beyond the Data – Build a Culture Focused on Data Quality


The 12 Directives: Set One: Recognize the importance of

data to the organization’s mission

• Directive 1: Obtain Management Commitment to Data Quality

• Current State Assessment Goal: To understand how management generally respond to strategic needs in order to identify obstacles to and opportunities for developing an organizational commitment to data quality improvement.

• Assess how your organization’s management works – review business plans, identify obstacles to success, talk with and formally survey the management team.

• Plan to communicate on an ongoing basis. Management commitment must be cultivated over time. Plan to talk their language: Success stories, cost/benefit analysis (CBA), results of other assessments.

• Directly relate business uses of data to the organization’s mission statement.

• Directive 2: Treat Data as an Asset

• Current State Assessment Goal : To understand how the organization currently describes the value of data and to turn the organization toward recognizing the value of data in monetary terms.

• Review operational and project budgets to define the current investment in data management and the cost of data issues to put the value of data in monetary terms.

• Identify best case and worst case scenarios for one strategic data set to quantify the benefits of improvement and the risks of poor quality data.

• As a pilot project, pick one improvement opportunity define its CBA directly in relation to the organization’s mission.


The 12 Directives: Set One: Recognize the importance of

data to the organization’s mission

• Directive 3: Apply Resources to Focus on Quality

• Current State Assessment Goal: To understand the organization’s readiness to

formally engage a team in the tactical execution of data quality improvement.

• Assess how the organization currently responds to data quality issues – review help

desk tickets, incident reports, break-fix projects.

• Survey teams to identify activities they currently engage in that support data quality.

Identify areas of redundant activity. Translate these to CBA.

• As with Directive 2, review operational and project budgets to define the current

investment in data management and the cost of data issues to put the value of data

in monetary terms.

• Directive 4: Build explicit knowledge of data

• Current State Assessment Goal: To understand the organization’s existing

knowledge-sharing practices in order to identify improvements that support

improvement of data quality.

• Identify existing processes for the creation and management of metadata, training

materials, and system documentation. Produce and inventory and identify gaps in

these areas.

• Survey teams about how they find answers to questions about data.


The 12 Directives: Set Two: Apply concepts related to

manufacturing physical goods to data

• Directive 5: Treat Data as a Product of Processes which can be Measured and Improved

• Current State Assessment Goal: To understand how your organization currently thinks about data in order to move the organization toward an understanding of data as a product so that product quality improvement practices succeed.

• Survey business and IT stakeholders about the condition of data.

• Catalog existing documentation of the data chain; identify gaps in knowledge (see Directive 4).

• Identify processes which are well defined and measured and find out how they became so. These can serve as examples of the impact of improvements.

• Directive 6: Recognize Quality is Defined by Data Consumers

• Current State Assessment Goals: To identify data consumers who can provide ongoing input about the quality of the organization’s data and how to improve it. And to identify which data is most critical or at risk within the organization.

• Survey data consumers about their needs, perceptions and concerns.

• Review access logs and reports to quantify which data elements are used most frequently. Review helpdesk tickets, issue logs, and break-fix projects to identify recurrent quality issues.

• Findings can serve as input to pilot improvement or measurement projects.


The 12 Directives: Set Two: Apply concepts related to

manufacturing physical goods to data

• Directive 7: Address the Root Causes of Data Problems

• Current State Assessment Goal: To understand the organization’s cultural approach to problem solving in general in order to identify obstacles and opportunities for advocating for root cause remediation.

• Identify (or diagram) the existing processes for remediation of data issues; including how issues are prioritized and how remediation is funded.

• Identify examples of data issues rooted in one system that impact the use of data in a different system.

• Perform a CBA on at least one example to quantify the benefits of root cause remediation.

• Directive 8: Measure Data Quality, Monitor Critical Data

• Current State Assessment Goal: To understand the organization’s current practices for data quality measurement and to identify options for implementing a set of pilot measurements.

• Identify a set of critical or at risk data elements; focus on data with recurrent issues.

• Apply DQAF (data quality assessment framework) measurement concepts to quantify issues relate to this data and produce a CBA.

• Propose a pilot for ongoing monitoring.


The 12 Directives: Set Three: Build a culture of quality

that can carry out strategic data management

• Directive 9: Hold Data Producers Accountable for the Quality their Data (and knowledge about that data)

• Current State Assessment Goal: To understand the organization’s general approach to accountability in order to identify opportunities for creating greater accountability for producing high quality data.

• Identify existing mechanisms for communication up and down the data chain. Assess obstacles to communication, gaps in communication, and instances where communication has worked well.

• Survey data producers for ideas on how they can improve their own processes.

• Incorporate data quality goals in to performance evaluations, service level agreements, and other formal mechanisms for accountability.

• Directive 10: Provide Data Consumers with the Knowledge the Require for Data Use

• Current State Assessment Goal: To understand how well informed data consumers are about the production and meaning of data and to identify ways that they educate themselves about data in order to identify opportunities for improvement.

• Use same assessments as under directive 4 and directive 6. Focus on how data consumers leverage or create explicit knowledge of data.

• Identify processes that encourage data consumers to actively learn about data rather than passively accept data in its current condition.

• Survey data consumers on how to improve their ability to understand and use data.


The 12 Directives: Set Three: Build a culture of quality

that can carry out strategic data management

• Directive 11: Data will continue to evolve – Plan for evolution

• Current State Assessment Goal: To understand the organization’s overall ability to

respond to emerging business opportunities and to identify ways to manage data so

that the organization can respond in the most agile ways possible.

• Identify major trends in your industry. How will your organization obtain or produce

the data needed to meet them?

• Identify risks within existing business and technical processes that could become

obstacles to future evolution. Use these examples to elicit suggestions about

approaches to effective planning.

• Directive 12: Data Quality goes Beyond the Data – Build a Culture

Focused on Data Quality

• Current State Assessment Goal: To identify and understand the effectiveness of

existing organizational structures that support improved data quality.

• Identify organizational components that are directly charged with data

management, data governance, or data quality, and survey the people involved

about what is working and what could be done better.


Current State Assessment Deliverables

• If you assess organizational readiness for 1 or all 12 of these

directives, you will have a set of findings and opinions and examples

that describe where your organization is and where it could go:

– Critical data; data that is less important.

– Processes that are working well; processes that are not working well.

– Insight on how your organization works:

• Behaviors that will enable improved quality

• Behaviors that will get in the way of improving quality

• This information is input to your formulation of a set of tactical actions

that can move your organization toward its strategic goal of improved

data quality.

– A set of prioritized recommendations

– Proposal for how to implement them



Classics of Information Quality

• English, Larry P. (1999). Improving Data Warehouse and Business Information Quality. Indianapolis, IN: Wiley.

• English, Larry P. (2009) Information Quality Applied. Indianapolis, IN: Wiley.

• Loshin, David. (2001). Enterprise Knowledge Management: The Data Quality Approach. Boston, MA: Morgan Kaufmann.

• Loshin, David. (2011). The Practitioner’s Guide to Data Quality Improvement. Boston, MA: Morgan Kaufmann.

• Maydanchik, Arkady. (2007). Data Quality Assessment. Bradley Beach, NJ: Technics Publications, LLC.

• McGilvray, Danette. (2008). Executing Data Quality Projects: Ten Steps to Quality Data and Trusted Information.™ Boston, MA: Morgan Kaufmann.

• Mosely, Mark, Brackett, Michael, Early, Susan, & Henderson, Deborah (eds.). (2009). The Data Management Body of Knowledge (DAMA-DMBOK Guide). Bradley Beach, NJ: Technics Publications, LLC.

• Olson, Jack. (2003). Data Quality: The Accuracy Dimension. Boston, MA: Morgan Kaufmann.

• Redman, Thomas C. (2008). Data Driven: Profiting from Your Most Important Business Asset. Boston, MA: Harvard Business Press.

• Redman, Thomas C. (2001). Data Quality: The Field Guide. Boston, MA: Digital Press.

• Redman, Thomas C. (1996). Data Quality for the Information Age. Boston, MA Artech House.

• Wang, Richard. (1998, February). A Product Perspective on Total Data Quality Management. Communications of the AMC. 58-65.

• Wang, Richard and Strong, Diane. (1996, Spring). Beyond Accuracy: What Data Quality Means to Customers. Journal of Management Information Systems. 5-33.


Recommended Reading – Thinkin’ differently about data

• Chisholm, Malcolm D. (2010). Definitions in Information Management: A Guide to the Fundamental Semantic Metadata. Canada: Design Media.

• Chisholm, Malcolm D. (2012-08-16) Data Quality is Not Fitness for Use. Information Management. http://www.information-management.com/news/data-quality-is-not-fitness-for-use-10023022-1.html

• Crease, Robert P. (2011). World in the Balance: The Historic Quest for an Absolute System of Measurement. New York: W. W. Norton Company.

• Derman, Emanuel. (2011). Models. Behaving. Badly.: Why Confusing Illusion With Reality can lead to Disaster on Wall Street and in Life. New York: Free Press.

• Gould, Stephen Jay. (1996). The Mismeasure of Man. New York, NY: Norton.

• Ivanov, Kristo. (1972). Quality-Control of Information: On the Concept of Accuracy of Information in Data-Banks and in Management Information Systems. Stockholm, Sweden: The Royal Institute of Technology and the University of Stockholm Sweden.

• Kent, William. (2000). Data and Reality. Bloomington, IN: 1st Books Library.

• Taleb, Nassim Nicholas. (2007). The Black Swan: The Impact of the Highly Improbable. New York, NY: Random House.

• Tufte, Edward R. (1983). The Visual Display of Quantitative Information. Cheshire, CT: Graphics Press.

• West, Matthew. (2011). Developing High Quality Data Models. Boston, MA: Morgan Kaufmann.

http://www.information-management.com/news/data-quality-is-not-fitness-for-use-10023022-1.html

Thank You

Laura Sebastian-Coleman, Ph.D., IQCP

Optum Insight

[email protected]

mailto:[email protected]