Validator

Advanced ETL Processor Data Validator guarantees to your business that every data value is correct and accurate. It checks the data against a set of user-defined rules and constraints and identifies any errors or inconsistencies that may be present. This helps organizations avoid problems such as incorrect data being loaded into their systems, or data being lost or corrupted during the ETL process

What is Data Validation

Data validation is the process of ensuring that data meets certain criteria or rules. It helps identify inaccuracies, inconsistencies, or anomalies within datasets. Here are some common ways to validate data:

1. Data Type Validation: Verifies that data is in the expected format and adheres to predefined data types. For example, ensuring that a field intended for numerical data only contains numbers.
2. Range and Boundary Validation: Checks if data falls within predetermined ranges or boundaries. It helps detect outliers or invalid values. For instance, validating that a temperature reading is within a specific range.
3. Format and Pattern Validation: Ensures that data conforms to a specified format or pattern. This is particularly useful when dealing with data such as email addresses, phone numbers, or postal codes.
4. Consistency Validation: Compares data across multiple sources or fields to identify discrepancies or conflicts. It helps maintain data integrity by ensuring consistency within datasets.
5. Referential Integrity Validation: Verifies that relationships between different data elements are maintained. For example, confirming that foreign key values in a database table exist in the referenced primary key column.

It does not matter which business you are in sooner or later you will discover that there is something wrong with the data and it has to be validated. Here when Advanced ETL Processor validator can help.

Note:

• Records can also be rejected by the Server.
• If there are several validation rules and one of them rejects the record and another discards it, the record will be discarded

Validator Screen

To change Validator properties double click on it

Tip:

If no data is present in the grid check the previous step execution log.

About input and output fields:

• List of Output fields is always the same as Inputs.
• List of Input fields is taken from the previous object

Validator Toolbar

1. Properties: View or modify object properties.
2. Save: Save Changes.
3. Create New Tab: Open a new workspace tab.
4. Cut: Remove the selected item and place it on the clipboard.
5. Copy: Duplicate the selected item to the clipboard.
6. Paste: Insert content from the clipboard.
7. Delete: Remove the selected item.
8. Redo: Reapply the last undone action.
9. Undo: Reverse the last action.
10. Align Left: Align selected objects to the left.
11. Arrange Vertically: Organize objects vertically.
12. Align Right: Align selected objects to the right.
13. Align Bottom: Align selected objects to the bottom.
14. Arrange Horizontally: Organize objects horizontally.
15. Align Top: Align selected objects to the top.
16. Space Horizontally: Distribute objects evenly across horizontal space.
17. Space Vertically: Distribute objects evenly across vertical space.
18. Snap to Grid/Show Grid: Enable grid snapping or display the grid.
19. Print Mapping: Print the current mapping layout.
20. Print Preview Mapping: Preview the mapping before printing.
21. Automap Data: Automatically map inputs to outputs.
22. Delete All Objects: Remove all objects from the workspace.
23. Delete All Links: Remove all connections between objects.
24. Search Objects: Locate specific objects in the workspace.
25. Process Data: Execute the data processing.
26. First Record: Navigate to the first record.
27. Previous Record: Move to the preceding record.
28. Next Record: Advance to the next record.
29. Last Record: Navigate to the last record.
30. Show Source Data: Display the source data.
31. Zoom In: Magnify the view of the workspace.
32. Zoom Out: Reduce the view of the workspace.
33. Zoom Back to 100%: Reset the workspace view to the default zoom level.

Note:

We recommend giving the Validator name, it makes it easier to identify problems later.

Debugging Validation

To start debugging validation press the Process Data button

To test data edit it in the Data grid.

To Change Validation Rule properties double click on it (or right click and select properties)

Note:

Use <value> to include actual value into default value

To add a new Validation rule drag and drop it from the Validation rules panel

It is also possible to apply several validation rules to the Input field by joining them

Validation Rules

• Validating Strings
• Validating Numbers
• Validating Dates
• Validating Times
• Miscellaneous Validations
• Validating Data using Regular Expressions

Video Tutorial

Posted on January 25, 2025 • 4 min read • 808 words