CRM Data Quality Management 2003 2003. 11. 11 (SK ) hskim226@skcorp.com
Why Quality Management? Prologue,,. Water Source Management 2 Low Quality Water 1) : High Quality Water 2) : ( ) Water Quality Management Change Management 1) 2) 2
Agenda I. Data Quality 1. Data Quality 2. 3. Data Quality II. Data Quality Management 1. Data Quality Management 2. Data Quality Management Framework III. Data Quality Management 1. DB 2. Data Enrichment 3. Data Process IV. SK Data Quality Management System V. Conclusion 1. Total Cost of Data Quality 2. 3
1. Data Quality I. Data Quality Data 10~25%, IT 40~50%, Data Quality. Data cost? Data Strategy Issue Little 4% Not sure 9% High 42% Business IT Data Quality Management. -IT Issue Database Performance 16% Management Expectations 17% Moderate 49% * Assential, Priority Learning - 100, 2001 Business Rule Analysis Legacy Data Transformation 22% 25% - Non quality data costs End-user Expectations 29% (Procession Cost) (Marketing Cost) Mission (Corporate Risk) Business Data Modeling Managing Data Quality 31% 46% * The Data Warehouse Institute - 1,670, 2000 4
1. Data Quality I. Data Quality Gartner, CRM Data (According to a report by Gartner, Inc., the no. 1 reason for CRM failures is ignoring the data) 7 Key Reasons Why CRM Fails, Gartner, Inc.Nelson & Kirkby, 2001 DW Data Quality [Information customer ] (Experience is revealing that more than half of data warehouse built fail to meet expectations because of poor information quality) Improving DW and Business IQ, Larry English, 1999 Data Warehousing Institute, 6,000 ( 720 ) Data Quality (The Data Warehousing Institute estimates that data quality problems cost U.S. businesses more than $600 billion a year) DQ and the Bottom Line, Wayne Eckerson, 2001 CRM Data Application,. Integrating disparate distributed data, applications, and legacy systems across multiple organizations is the single most significant barrier to enterprise CRM execution Research Report, Meta Group, 2000 5
2. Data I. Data Quality LTV, Mission., LTV Mission. (Scrap & Rework ) Data (Cleansing ) / Data Process Failure Non quality Data (DM/TM ) Non quality Data, Mission Mission 6
2. Data : 1 I. Data Quality Data Mining Sampling ~ Modification 50~70%, Modeling, Model Quality. Data Mining Process < Risk of Non-quality Data > D/W D/W Sampling Exploration Modification Data Data Scrap Data Data Modeling Assessment Data Model Model.. 7
2. Data : 2 I. Data Quality Slang,. Communication : : 1111 Slang D/W Slang D/W Slang 8
3. Data Quality I. Data Quality Consistently meeting all knowledge worker and end-customer expectations through Data and Data services to accomplish enterprise and customer objectives - Larry English Consistently : Data Quality Data Meeting : Data, Needs. Knowledge Worker & End Customer : Data Data Quality Information Customer Expectations : Information Quality 9
Agenda I. Data Quality 1. Data Quality 2. 3. Data Quality II. Data Quality Management 1. Data Quality Management 2. Data Quality Management Framework III. Data Quality Management 1. DB 2. Data Enrichment 3. Data Process IV. SK Data Quality Management System V. Conclusion 1. Total Cost of Data Quality 2. 10
1. Data Quality Management II. Data Quality Management Data Quality Management, /. DB / Data / Data Enrichment / Data Data Process Data Quality Management System 11
2. Data Quality Management Framework II. Data Quality Management Data Quality Management DB Data Enrichment, Data Process, Data Quality Management System. Data Quality Management DB Data Enrichment Data Process Needs Enrichment DB / / - - Enrichment Process Data Process Data Quality Management System Merge/Purge Tool DataStat,, Tool 12
Agenda I. Data Quality 1. Data Quality 2. 3. Data Quality II. Data Quality Management 1. Data Quality Management 2. Data Quality Management Framework III. Data Quality Management 1. DB 2. Data Enrichment 3. Data Process IV. SK Data Quality Management System V. Conclusion 1. Total Cost of Data Quality 2. 13
1. DB III. Data Quality Management Needs 14
2. Data Enrichment III. Data Quality Management Data Enrichment Data, Source /. Data Enrichment Data Enrichment Data Enrichment 3.,. Source / / /, 15
2. Data Enrichment III. Data Quality Management Data Enrichment Data Needs Segment,,, Issue Data Enrichment Cycle. Data Enrichment Data Needs Data Needs. Data Enrichment Needs, / Enrichment Segment,. Data Enrichment Enrichment. Enrichment / Data Data Enrichment. Feedback Data Enrichment, Enrichment Issue Data Enrichment. 16
2. : Data Enrichment III. Data Quality Management 17
3. Data Process III. Data Quality Management Data Process Data DW Knowledge Worker, / ECTL Data. Data Warehouse Strategic/Tactical Knowledge Worker ECTL* Repository/ Data Dictionary Business Data Customer Operational Knowledge Worker External Data Information Producer * ECTL : Extract, Correction, Transformation, Loading 18
3. Data Process III. Data Quality Management 19
Agenda I. Data Quality 1. Data Quality 2. 3. Data Quality II. Data Quality Management 1. Data Quality Management 2. Data Quality Management Framework III. Data Quality Management 1. DB 2. Data Enrichment 3. Data Process IV. SK Data Quality Management System V. Conclusion 1. Total Cost of Data Quality 2. 20
1. Merge/Purge System IV. SK Data Quality Management System Merge/Purge,, 3. FTP : 21
1. Merge/Purge System : IV. SK Data Quality Management System M/P / Data, Data process. [ M/P ] [ M/P Data check ] M/P M/P (%) 229,952 3,570,960 1 3,461,797 2 1,505,547 1 1,473,622 2 4,043,207 4,043,207 2,442,159 371,800 0 0-100 60.4 5.69 9.19 88.32 85.62 0 37.24 36.45 0 M/P M/P (%) 4,043,207 3,879,217 2,116,827 190,973 384,761 3,222,352 3,225,030 1,909,486 291,000 291,345 213,542-95.94 52.35 4.72 9.52 79.7 79.76 47.23 7.2 7.2 5.28 0 163,990 325,332 38,979-12,961 348,608 236,767-1,909,486 1,214,547 1,182,277-213,542 (3,4 ) (3,4 ) Example 3,879,217 14,018 149,433 527 12 2,116,827 10,128 204,062 71,272 39,743 190,973 6,686 8,907 6,727 12,504 22
2. DataStat IV. SK Data Quality Management System / Reference DB / *. Example 23
3. COSMOS : Column Statistics Monitoring System IV. SK Data Quality Management System DB Column Total Count, Not Null count, DB Data. - Numeric Data Min, Max, Average, Sum,, 0, Count,. - Date Data (Invalid Count), Min, Max,, - Code Data Domain, Domain Data. Example 24
4. Meta IV. SK Data Quality Management System Meta, Data, Biz Rule, Data,, Data. Meta DB, Biz Rule, Type, Data.,. Example 25
Agenda I. Data Quality 1. Data Quality 2. 3. Data Quality II. Data Quality Management 1. Data Quality Management 2. Data Quality Management Framework III. Data Quality Management 1. DB 2. Data Enrichment 3. Data Process IV. SK Data Quality Management System V. Conclusion 1. Total Cost of Data Quality 2. 26
1. Total Cost of Data Quality Management V. Conclusion DQM., Awakening Data Danger Point. Danger Point Data Quality Cost Information Scrap & rework & Process Failure Costs Data Correction Costs Assessment Costs IQ Improvement, Environment Investments Stage 1 Uncertainty Stage 2 Awakening Stage 3 Enlightenment Time Stage 4 Wisdom Stage 5 Certainty information quality information quality? information quality,. Information quality. Information Quality. 27
2. V. Conclusion DQM Task :. /,. Start Now :.(CRM ) 28
App.1 : Ten Essential Ingredients of Information Quality Management Appendix 1. Understand information quality is a business problem, not just a system problem; and solve it as a business process, not just as a systems process. 2. Focus on the information customers and suppliers, not just the data. 3. Focus on all components of information, including definition, content and presentation. 4. Implement information quality management processes, not just information quality software. 5. Measure data accuracy, not just validity. 6. Measure costs -not just percent- of non-quality information and business results of quality information. 7. Emphasize process improvement and preventive maintenance(plan-do-check-act), not just corrective maintenance (data cleansing). 8. Improve processes at the source, not just in downstream business areas. 9. Provide quality training to managers and information producers (who are their information customers and what do they need?). 10. Actively transform the culture, don t just implement activities. 29
App.2: The 13 th Information Quality Conference Appendix 1. 2. 30
App.2: The 13 th Information Quality Conference - Appendix 3. Keynotes 31
Q & A 32