11 May Data Vault structure for SCV
Why are Single Customer View projects so hard?
Single Customer View projects traditionally seek to create a single source of truth about customers and their touch points with an organisation. The project attempts to line up data from all the different sources in the load process assuming that customer data will form one tightly integrated result. It all sounds straightforward – surely customer data is the same whatever the source?
But many organisations fail to deliver on the promise. The projects take too long, get bogged down in technical details, drop entire feeds, or are cancelled. What could be going on?
The reality is that customer data tends to be all pervasive – handled by many departments and held in many different systems. Think in-house systems such as Sales Order Processing, Financials, CRM, Customer Support, EPOS, Call Centre, Loyalty etc…. then there are external social media applications – eCommerce, marketing automation or other cloud applications. Different systems use different customer identifiers. There are matching challenges made worse by data quality, consistency and semantic issues. How many duplicates are there in your systems? How many spelling mistakes do your call centre agents make when typing names? How many different address formats are used? Has data been verified? Do we know under what circumstances a phone number may be used? When is a match a match?
The issues multiply – leading to paralysis.
Part of the problem is that traditional approaches to building these systems attempt to process and standardise data as it is loaded into the Single Customer View warehouse. All the issues of matching, data cleaning, formatting, etc. need to be solved and coded before we can load the data and use it.
A Data Vault system is designed to ingest all source data and not to try and force data quality cleaning, matching and other processing on load
There is a better way than this tightly integrated approach to data. The Data Vault method takes a loosely federated approach to customer data. A Data Vault system is designed to ingest all source data and not to try and force data quality cleaning, matching and other processing on load. Data quality issues are addressed strategically in the Data Warehouse itself using Business Rules. We never lose the source data so we can work on incremental improvements to our Business Rules – and even change our mind about the rules and simply recalculate the results. Data quality is a business problem not a technical one!
Data quality is a business problem not a technical one
This allows your Single Customer View project to be fast and agile. Data streams can be added one at a time and Business Rules refined as our understanding of the data matures.