Directory Monitor Issue

We are still looking at it. I am sorry that it is taking so long

 

I have spent entire day trying to reproduce the problem and I was not able to do so. It just does not fail here. 
We did publish another release with more detailed logging and some minor tweaks  
I've also spent time reading documentation and found that although monitoring network shares works it is unreliable.
You did say that previous version works absolutely fine.
It is been a very long time since we modified it.
It is strange that the interval between duplicated event records is 4 minutes. 
It looks like genuine event to me. Question is how and why it was created.
Perhaps it is a result of antivirus scan. I can only guess.

Regarding your questions about the log.

There are multiple waModified events in the log because files are copied in chunks especially when copying files across  the network
AETL catches all events, filters out duplicated events  and filters out not matching events.

I hope that helps

01/07/2025 12:16:42    DBSLAMD    dbsl    69    \\192.168.1.122\Shared    Fx Forward Curves_OneBusinessDayAgo_26_06_25.csv    waAdded    position*.csv    0
01/07/2025 12:16:42    DBSLAMD    dbsl    70    \\192.168.1.122\Shared    Fx Forward Curves_OneBusinessDayAgo_26_06_25.csv    waModified    position*.csv    0
01/07/2025 12:16:42    DBSLAMD    dbsl    71    \\192.168.1.122\Shared    Fx Forward Curves_OneBusinessDayAgo_26_06_25.csv    waModified    position*.csv    0
01/07/2025 12:16:42    DBSLAMD    dbsl    Duplicated Event
01/07/2025 12:16:46    DBSLAMD    dbsl    72    \\192.168.1.122\Shared    Positions Today_26_06_25.csv    waAdded    position*.csv    0
01/07/2025 12:16:46    DBSLAMD    dbsl    Matched :Positions Today_26_06_25.csv, Mask:position*.csv
01/07/2025 12:16:46    DBSLAMD    dbsl    73    \\192.168.1.122\Shared    Positions Today_26_06_25.csv    waModified    position*.csv    0
01/07/2025 12:16:47    DBSLAMD    dbsl    74    \\192.168.1.122\Shared    Positions Today_26_06_25.csv    waModified    position*.csv    0
01/07/2025 12:16:47    DBSLAMD    dbsl    Duplicated Event
01/07/2025 12:16:47    DBSLAMD    dbsl    File ready: \\192.168.1.122\Shared\Positions Today_26_06_25.csv
01/07/2025 12:16:49    DBSLAMD    dbsl    75    \\192.168.1.122\Shared    Pricing History_OneBusinessDayAgo_26_06_25.csv    waAdded    position*.csv    0
01/07/2025 12:16:49    DBSLAMD    dbsl    76    \\192.168.1.122\Shared    Pricing History_OneBusinessDayAgo_26_06_25.csv    waModified    position*.csv    0
01/07/2025 12:16:49    DBSLAMD    dbsl    77    \\192.168.1.122\Shared    Pricing History_OneBusinessDayAgo_26_06_25.csv    waModified    position*.csv    0
01/07/2025 12:16:49    DBSLAMD    dbsl    Duplicated Event
01/07/2025 12:16:51    DBSLAMD    dbsl    78    \\192.168.1.122\Shared    TGM_PMA_062425094359.csv    waAdded    position*.csv    0
01/07/2025 12:16:51    DBSLAMD    dbsl    79    \\192.168.1.122\Shared    TGM_PMA_062425094359.csv    waModified    position*.csv    0
01/07/2025 12:16:51    DBSLAMD    dbsl    80    \\192.168.1.122\Shared    TGM_PMA_062425094359.csv    waModified    position*.csv    0
01/07/2025 12:16:51    DBSLAMD    dbsl    Duplicated Event
01/07/2025 12:16:53    DBSLAMD    dbsl    81    \\192.168.1.122\Shared    TGM_PMA_062625093917.csv    waAdded    position*.csv    0
01/07/2025 12:16:53    DBSLAMD    dbsl    82    \\192.168.1.122\Shared    TGM_PMA_062625093917.csv    waModified    position*.csv    0
01/07/2025 12:16:53    DBSLAMD    dbsl    83    \\192.168.1.122\Shared    TGM_PMA_062625093917.csv    waModified    position*.csv    0
01/07/2025 12:16:53    DBSLAMD    dbsl    Duplicated Event
01/07/2025 12:23:12    DBSLAMD    dbsl    84    \\192.168.1.122\Shared    site-map.txt    waAdded    position*.csv    0
01/07/2025 12:23:12    DBSLAMD    dbsl    85    \\192.168.1.122\Shared    site-map.txt    waModified    position*.csv    0
01/07/2025 12:23:12    DBSLAMD    dbsl    86    \\192.168.1.122\Shared    site-map.txt    waModified    position*.csv    0
01/07/2025 12:23:12    DBSLAMD    dbsl    Duplicated Event
01/07/2025 12:24:42    DBSLAMD    dbsl    87    \\192.168.1.122\Shared    site-map.txt    waRemoved    position*.csv    0


 

Hi Team and thanks for the update and info. I'm afraid this is causing some pain my side and I now having to consider workarounds. But to at least add something. I have installed .67 and can see the extra logging which I have provided. I have tested both network and non-network locations. and also included a sample at 1:50PM today AEST where it was monitoring a local drive and still created 2 queue

entries 
 

In a separate test using different packages

* I emptied the log
* Dropped a NON-MATCHING file into folder (network)
* The file was noticed but did not trigger the package as workflow as expected
* Dropped a MATCHING file in the folder
* The file was noticed and then 2 Entered were created in the queue within nano-seconds of each other. 

I have reviewed and included the log. IMHO it appears to have noticed exactly what it should have and that the issue is not the monitor part but the part that then decides to add the queue entry or not. 

Please note that the 2 rows you see in the execution log is not 1 for the NON-MATCHING file and 1 for the MATCHING file. They are BOTH for the MATCHING file. The NON-MATCHING file was ignored as expected.

I also thought that I may have had it. All of the packages except for this test one MOVE the inbound trigger file(s) and if I drop that step then the duplicate step does not happen. This may e a red herring as my cutdown test setup doesn't have any file operations.

As you say monitoring a network share seems to be problematic. Although it has never been before. But in my first sample above it was a local folder and file


 
 

Can you attach the actual log file it is not possible to read the image I am afraid

For C*.csv file can you provide a screeshot of directory monitor settings please