Pattern Validation

Practical data validation articles for ETL workflows, source files, and automated data quality checks.

Advanced ETL Processor
4.9 ★★★★★ Based on 16 reviews on Capterra See all reviews on Capterra →

Pattern validation checks whether a value follows a required text pattern. Advanced ETL Processor Enterprise validates postcodes, product codes, customer IDs, invoice numbers, phone numbers, and other formatted fields inside a visual ETL workflow. It rejects or routes failed rows, logs the failure reason, and keeps the original source data unchanged. The data may look tidy and still be wrong, which is very on-brand for source files.

Pattern validation in Advanced ETL Processor

Patterns catch values that look almost right

Pattern validation uses a rule or regular expression to check the structure of a field. It does not ask whether a postcode exists in the real world. It checks whether the value follows the expected format before the row continues through the workflow.

That distinction matters. A postcode can be the right type, the right length, and still not match the format the business expects. Pattern validation catches those rows early, before they reach reporting, shipping, billing, or customer systems.

UK postcode format validation is a common pattern check

A UK postcode follows a recognisable outward and inward code structure, such as SW1A 1AA, M1 1AE, or EC1A 1BB. The exact regular expression depends on how strict the workflow needs to be.

  • Valid pattern: SW1A 1AA.
  • Invalid pattern: 12345.
  • Review pattern: SW1A1AA, if your rule requires the space.

The space is a small thing until a downstream system treats it like a personal insult. Databases can be sensitive creatures.

Pattern validation is useful beyond postcodes

  • Customer account numbers must follow the approved prefix and number structure.
  • Invoice numbers must match the finance team's format.
  • Product SKUs must use allowed letters, digits, and separators.
  • National identifiers must follow the expected pattern before further validation.
  • File names must match the import convention before scheduled processing starts.

Use regular expressions carefully

Regular expressions are powerful, but they become unreadable quickly. Write the business rule in plain English first, then build the pattern. Test it with valid examples, invalid examples, and awkward examples that only appear on Friday afternoons.

Do not make the pattern stricter than the business needs. If the source sometimes sends lowercase postcodes or missing spaces, decide whether to standardise the value or reject the row. Both approaches are valid when they are deliberate. Guessing silently is the risky option.

Useful references

Related checks include field length validation, date format validation, and creating complex data validation rules. For setup details, see the Validator object documentation and the GOV.UK guide to contact information standards.

Video

Next steps

If this validation rule belongs in a repeatable ETL process, configure it in the Validator object, test it on a sample file, then decide whether failed rows should be rejected, logged, or routed for review.

Keep the original data unchanged. Raw files are still the witness statement.