Addressing Data Quality Issues can be split into three simple steps:
- Define Data Quality Rules
- Perform data validation and transformation
- Report on found issues
Advanced ETL Processor can be used to perform the first two, It has 300+ validation/transformation functions plus it works with 27 data sources. For the third one, you can use QlikView.
Everyone who performs data transformation knows the importance of data quality and how much time is required to find what went wrong and correct the problem
In the latest version of Advanced ETL Processor, we introduced a new object called "Log"
The idea is very simple: to redirect log messages into different data flow so this data can be used later for reporting.
Plus compared to standard text log it provides much more information to the user.
It allows you to answer the following questions:
What was wrong with the data?
Which actions were taken to correct the data?
An initial value of the field
Value after correction
Source file/table
Record number
Date
Computer name
User name
Customer Name
Person responsible
Date
And much more
QlikView Data Quality Dashboard
We would like to thank Adrian Parker from Differentia Consulting Ltd for giving us useful feedback
