knowledgebase:processing_files_based_on_modification_date

Processing files based on modification date

I would like to pick a specific file on a different server at a particular time if it was modified in the last 2 hrs and load data from it into the database. If the file was not updated in the last 2 hrs a mail needs to be sent to the responsible personnel.

There are several ways of doing it, as usual, you can write a script to check the modification date or you can use transformations for it. Transformation is an easier option. Create two transformations First, one will be used to check file modification date Second one for loading the data.

Set DataReader source type to File System and enter the file name into the mask box.

Open Validator and create validation as in the picture below The logic is very simple We use to compare the value to check if modified date is less than two hours ago and if it is we set the result to true Then we use Is Equal to abort execution

The file might be locked for editing so we try to copy it first

For more technologies supported by our ETL Software see Advanced ETL Processor Versions and Visual Importer ETL Versions

Confused? Ask question on our ETL forum

  • knowledgebase/processing_files_based_on_modification_date.txt
  • Last modified: 14/07/2021 09:01
  • by admin