Processing files based on modification date

Question from the customer

I would like to pick a specific file on a different server at a particular time if it was modified in the last 2 hrs and load data from it into the database. If the file was not updated in the last 2 hrs a mail needs to be sent to the responsible personnel.

Solution

There are several ways of doing it, as usual, you can write a script to check the modification date or you can use transformations for it. Transformation is an easier option. Create two transformations First, one will be used to check file modification date Second one for loading the data.

Checking file modification date

Set DataReader source type to File System and enter the file name into the mask box.

Data Reader Properties

Transformation

Open Validator and create validation as in the picture below The logic is very simple We use to compare the value to check if modified date is less than two hours ago and if it is we set the result to true Then we use Is Equal to abort execution

Validator

Objects Properties

Change Date Properties

Change Value Properties

Is Equal To Properties

Package

Package

For more technologies supported by our ETL Software see Advanced ETL Processor Versions

Confused? Ask question on our ETL Forum
Last updated: March 14, 2023