Addressing Data Qualtity Issues can be split into three simple steps:
- Define Data Qualtity Rules
- Perform data validation and transformation
- Report on found issues
Advanced ETL Processor can be used to perform first two, It has 300+ validation/tranformation functions plus it works with 27 datasources. For the third one you can use QlikView.
Everyone who performs data transformation knows the importance of data quality and how much time is required to find what went wrong and correct the problem
In the latest version of Advanced ETL Processor we introduced new object called "Log"
The idea is very simple: to redirect log messages into different data flow so this data can used later for reporting.
Plus comparing to standard text log it provides much more information to the user.
It allows to answer the following questions:
What was wrong with the data?
Which actions were taken to correct the data?
Initial value of the field
Value after correction
And much more
QlikView Data Quality Dashboard
We would like to thank Adrian Parker from Differentia Consulting Ltd for giving us usefull feedback