05 Oct Common pitfalls experienced in Data Vault 2.0 projects
Common pitfalls experienced in Data Vault 2.0 projects
Data Vault 2.0 is gaining traction as a method for the design and build of Data Warehouses but, as when you try anything for the first time there are some potential pitfalls. We have seen some issues our clients have faced, and we thought we would share three common pitfalls experienced in Data Vault projects and some tips for avoiding them.
Data Vault 2.0 is gaining traction as a method for the design and build of Data Warehouses
- Modelling without thinking
Data modelling is a craft and you get better with experience. There is no right or wrong way. It is about how you intend the model to be used: the context is important. Depending upon the analyst there may get different answers, for example modeller may make decisions early on about using specific entities such as customer and supplier rather than generic party entities. Many projects start down the wrong road and may need to reverse some decisions. How do you handle The good news is that Data Vault 2.0 embraces agile and makes it easier to refactor than other approaches. But it still takes time and it is better to be mostly right from the start. These issues can easily be mitigated by getting someone on the team with some experience of Data Vault 2.0 who can review models along the way.
- Applying standards
You are experienced in Data Warehouse design. You read Dan Linstedt’s book and you start work. There are a set of standards for Data Vault 2.0 projects that have been developed across many use cases in different industries for some years. Time and again we have seen capable and experienced Data Warehouse professionals discount some of the standards before they have a complete understanding of why they are there. Unfortunately, the consequences of not following standards may not become clear until later. An example of this might be putting links on links or effective date ranges on links. This may look sensible at the time, but it affects how a system scales. This can be mitigated by attending Data Vault 2.0 certification training where you can learn why the standards are applied and know when to flex them.
- Start simple and build out
There is often management pressure for results and so the first project selected is often too ambitious. Care should be taken when selecting the first project. We may want to replicate an existing report and show that it reconciles. For example, a sales report may require details of all transactions, all products, organisation structures, geographic structures and customers. Before you know it, you need data from 15 tables across 4 source systems and you have a very complex piece of work feeding from lots of systems. It might be better to select a part of a bigger reporting set. Often management or users can find surprising value in a subset of data. Replicating a well selected report can add considerable value and demonstrate the approach works. Speaking from experience, selecting the first project requires careful expectation management but can create significant positive momentum.
Get someone experienced to help get you started
We have seen different pitfalls across many clients, but these are some of the more common ones. Being aware of the common pitfalls experienced in Data Vault projects will improve your chances of success. We would recommend the getting someone experienced to help get you started. This may be hiring someone with experience into your team or simply buying some consulting time as a shortcut to getting it right from the start.
If you think you need some help don’t hesitate to get in touch with us.