Directory Monitor Issue

Hi Team

AETLE x64 v6.4.5.63

I have come an issue in Production which I can replicate. In summary when the Directory monitor detects a file is being added to the directory 2 queue entries are being created within the same second. I can tell this because the execution log shows them and I have a ShowMessage in the Package that displays the filename and the queue ID. The filename is the same, but the queue IDs are incremented by 1.

Secondly if I trigger the same process again by first removing the trigger file and then adding it again. The same thing happens except that in one instance the agent tries to run a transformation package with the same name instead. Which does not even exist. This can be seen in the screen shots attached. I might add that the parent log indicates that some sort of Boolean test is being done against an empty string which is also throwing an error. None of this occurs in my test package as it is a simple Show Message with the DirectorymonitorFilename variable and Queue ID variable

Even more odd is that from that point on the trigger seems to function correctly. However, that is not always the case and at this stage I am not sure what will cause a repeat other than a restart of the service.

It seems to be that certainly the "First" trigger event causes an issue. I am not sure when "First" is. Is it the first time in the hour, day or when the last snapshot was taken as a comparison?

Would appreciate some assistance as this is affecting our production platform. I am continuing to look for a workaround. and will advise if I find something. I am using the file MASK for the event monitor. 

To replicate this

• Stop the Directory Monitor Service
• Create a simple package that shows a message (I know it will get logged and not show interactively). The message should contain some runtime variables such as Queue ID and DirectoryMonitorFilename
• Create a monitor on an empty directory based on a wildcard such as *.tst
• Start the Directory Monitor Service
• Then create a file in that directory. I typically create a zero-byte file in another folder and copy it over. this allows me to repeat the trigger and test

In my case the above will cause the single file to create 2 agent queue entries for the same file and will execute them asynchronously. Meaning that if the package was to move the file into another folder, the second instance would fail - because the file is already there unless overwrite is selected, or it may fail due to a file lock etc. 

tell me what you need, and I will see if I can provide. 


   

Thanks again
Bruce



 

Looking at it now
 

Fixed it.

It is a compiler bug. I have never seen anything like this before.
Basically the variable was assigned incorrectly.
Only effects 64 bit version when CPU optimization is on.

Tweaked the code slightly and it started to work.

Everyday I learn something new.

Thank you for your feedback anyway.

 

Thanks again for the fantastic turn around time. Verified new version now works as expected.

Cheers

Hi

Seems that there is still some sort of related issue. It is not for all of our inbound files, but it is consistently from one source which is an on prem trading platform (on a different server and network). I am wondering if you have come across this before?

The trading platform is scheduled to run a report that produces a file that is dumped in a network share. The monitor is set to wait for 5 seconds for the file to be ready. In the new attachment you will see that the event creates 2 queue entries for the same file within a second of each other. This event is triggered by a File Add.

What is odd also is that the Submit times are roughly 1 second apart. However, the start and finish times are identical. Therefore, the scenario is that 2 queue entries were created and then after the second one is created, they both start executing. But because the operation is to move the file, the second instance fails - as the file is no longer there.

I am wondering if the trading platform starts creating the file, 5 seconds elapses, AETL considers the file close, creates the queue entry, but then somehow the platform closes the file which then results in the monitor creatin a second queue entry for the same file. Even that doesn't feel right and not something we can control.

I can change the package to not fail if the file is not there. But to me that is hiding the issue.  In the second example, different package but same source but different export folder).. There submit, start and finish times are different. But it is the same file that triggered the package. I can tell this from the log.

I might also add, that me copying & pasting files in and out of the folder to trigger the event works every time. I cannot reproduce the generation of the 2nd queue entry for the same file


 
 
 

Please check "Log events" and attach the log here