Data Validation

The Advanced ETL Processor has a robust validation process built in. The types and nature of the validations taking place can be tweaked and configured by the user. A full range of validation functions are included. Validations can be performed on the basis of data type, lists of values, and regular expressions, which can be individually changed according to requirements.

Summary of Validation Processes:

Checking quality of the data

 Advanced ETL Processor is able to check any data, for exampe date formats, post codes, phone numbers etc. It is also possible to write your own Regular expressions

 

Checking File format

Checking source data

 

What are the regular expressions?

Regular expression is a string that is used to describe or match a set of strings, according to certain syntax rules. Regular expressions are used by many text editors, utilities, and programming languages to search and manipulate text based on patterns. For example, Perl and Tcl have a powerful regular expression engine built directly into their syntax. Several utilities provided by Unix distributions—including the editor ed and the filter grep—were the first to popularize the concept of regular expressions.
"Regular expression" is often shortened to regex or regexp (singular), or regexes, regexps, or regexen (plural). Some authors distinguish between regular expression and abbreviated forms such as regex, restricting the former to true regular expressions, which describe regular languages, while using the latter for any regular expression-like pattern, including those that describe languages that are not regular. As only some authors observe this distinction, it is not safe to rely upon it. As an example of the syntax, the regular expression \bex can be used to search for all instances of the string "ex" that occur at word boundaries (signified by the \b). Thus in the string, "Texts for experts," \bex matches the "ex" in "experts," but not in "Texts" (because the "ex" occurs inside the word there and not immediately after a word boundary).

Source: wikipedia.

Below is a list of Date Validation Functions supported by Advanced ETL Processor

ImageData Validation FunctionDescription
Is Date Checks if the data is a Date
Is Date Between Checks if the data is a Date between two values
Is Date Less Than Checks if the data is a Date less than value
Is Date More Than Checks if the data is a Date more than value
Is Monday, Is Tuesday, Is Wednesday, Is Thursday, Is Friday, Is Saturday, Is Sunday, Is Today, Is Yesterday, Is Tomorrow, Is Weekend, Is Weekday Checks if the data is a Date value which belongs to specific day of week
Is January, Is February, Is March, Is April, Is May, Is June, Is July, Is August, Is September, Is October, Is November, Is December, Is Current Month, Is Last Month, Is Next Month Checks if the data is a Date value which belongs to specific month of a year
Is 1st Quarter, Is 2nd Quarter, Is 3rd Quarter, Is 4th Quarter, Is Current Quarter,Is Last Quarter, Is Next Quarter Checks if the data is a Date value which belongs to specific quarter of a year
Is Leap Year, Is Current Year, Is Last Year, Is Next Year Checks if the data is a Date value which belongs to specific year
Is Current Week, Is Last Week, Is Next Week Checks if the data is a Date value which belongs to specific week
Is Within Past Minutes, Is Within Past Hours, Is Within Past Days, Is Within Past Weeks, Is Within Past Months Checks if the data is a Date value within in past minutes etc.

Below is a list of Number Validation Functions supported by Advanced ETL Processor

ImageData Validation FunctionDescription
Is Number Checks if the data is a Number
Is Integer Checks if the data is an Integer
Is Positive Checks if the data is a Positive Number
Is Negative Checks if the data is a Negative Number
Is Number Between Checks if the data is a number between two values
Is Number Less Than Checks if the data is a number less than value
Is Number More Than Checks if the data is a number more than value

Below is a list of String Validation Functions supported by Advanced ETL Processor

ImageData Validation FunctionDescription
Is Null Checks if the data is equal to null.
Is Empty String Checks if the data is Empty String. (Length of the string is zero).
Is Alpha Checks if the data contains only Alpha characters.
Is Alpha Numeric Checks if the data contains only Alpha Numeric characters.
Is Hex Checks if the data contains valid HEX string
Is Equal To Checks if the data is equal to supplied value
Is Starts With Checks if the data starts with supplied value
Is Ends With Checks if the data ends with supplied value
Contains Checks if the data contains supplied value
In List Checks if the data exists within supplied list values

Below is list of Time Validation Functions supported by Advanced ETL Processor

ImageData Validation FunctionDescription
Is Time Checks if the data is a time value
Is Time Between Checks if the data is a time between two values
Is Time Less Than Checks if the data is a time less than value
Is Time More Than Checks if the data is a time more than value
Is Second Checks if data is a number between 0 and 59
Is Minute Checks if data is a number between 0 and 59
Is Hour 24 Checks if data is a number between 0 and 23
Is Hour 12 Checks if data is a number between 0 and 12
Is PM Checks if the data is a PM Date
Is AM Checks if the data is a AM Date

We have 301 guests and no members online