Addressing Data Quality Issues can be split into three simple steps:

  1. Define Data Quality Rules
  2. Perform data validation and transformation
  3. Report on found issues

Advanced ETL Processor can be used to perform the first two, It has 300+ validation/transformation functions plus it works with 27 data sources. For the third one, you can use QlikView.

Everyone who performs data transformation knows the importance of data quality and how much time is required to find what went wrong and correct the problem

In the latest version of Advanced ETL Processor, we introduced a new object called "Log"

Data Quality LogThe idea is very simple: to redirect log messages into different data flow so this data can be used later for reporting.

Plus compared to standard text log it provides much more information to the user.

It allows you to answer the following questions:

What was wrong with the data?
Which actions were taken to correct the data?
An initial value of the field
Value after correction
Source file/table
Record number
Computer name
User name
Customer Name
Person responsible
And much more

QlikView Data Quality Dashboard

QlikView Data Quality Dashboard

We would like to thank Adrian Parker from Differentia Consulting Ltd for giving us useful feedback

Direct link, no registration required.