Processing files based on modification date

Question from the customer:

I would like to pick a specific file on a different server at a particular time if it was modified in the last 2 hrs and load data from it into the database. If the file was not updated in the last 2 hrs a mail needs to be sent to the responsible personnel.

Solution:

There are several ways fo doing it, as usual, you can write a script to check modification date or you can use transformations for it. Transformation is an easier option. Create two transformations First, one will be used to check file modification date Second one for loading the data.

Checking file modification date

Set DataReader source type to File System and enter file name into the mask box.

Open Validator and create validation as on picture below The logic is very simple We use to compare the value to check if modified date is less than two hours ago and if it is we set result to true Than we use Is Equal to abort execution

Objects Properties

Package

The file might be locked for editing so we try to copy it first

For more technologies supported by our ETL Software see Advanced ETL Processor Versions and Visual Importer ETL Versions

Confused? Ask question on our ETL forum

  • knowledgebase/processing_files_based_on_modification_date.txt
  • Last modified: 13/04/2019 17:24
  • by admin