Directory Monitor Issue

More
1 year 1 month ago #25335 by bruce.gibbins
Thanks. Just spotted that. I have activated. But since I can't control the trading platform I will need to wait until it generates the next set of reports which may be tomorrow AEST

Please Log in or Create an account to join the conversation.

More
1 year 1 month ago #25336 by bruce.gibbins
Hi Team. Please find attached the LogEvents associated with the Event Monitor Package that seems to get triggered twice a few minutes apart. There are 4 event monitors polling the same folder but for different file names. I suspect this may have something to do with it. I have also attached an execution log screenshot showing one of the events getting triggered twice for the same file but a few minutes apart. I know it is the same file because I log the name in the package workflow 

Cheers
Attachments:
The following user(s) said Thank You: admin

Please Log in or Create an account to join the conversation.

More
1 year 1 month ago - 1 year 1 month ago #25341 by admin
Replied by admin on topic Directory Monitor Issue
Can you try latest version please.
I think it is better to do it on Monday when we are available for help.

 

Mike
ETL Architect
Last edit: 1 year 1 month ago by admin.

Please Log in or Create an account to join the conversation.

More
1 year 1 month ago - 1 year 1 month ago #25352 by bruce.gibbins
Hi Mike & Team. Issue persists after the upgrade which I had to do, due to the time difference, I needed to see if the problem had been resolved. I have included a screenshot of part of the directory monitor log file which is confusing to me and is either part of the problem or a different logging problem. But to clarify
  • The samples I have provided are for a single directory being monitored. However, it is across the board that once at one file is found two queue entries are found second queue event is created. I did test this with the current version and it was working fine. But this was with me copying files around folders. These issues are occuring for nearly if not all all files landing in montitored folders.
  • The Trading platform is configured to drop reports into that folder and it is possible that multiple files for different reports may arrive at the same time. Different reports with different names.
  • There is an EVent Monitor for each report which is based on a report filename wildcard because part of the filename is a date.
  • The consistent theme is that even though a single file is generated. This results in 2 ETL Queue entries for the same file
  • We have directory monitors on other folders and don't seem to have this issue. However, the source of files is different, and the folder is not shared with other file types
Attachments:
Last edit: 1 year 1 month ago by bruce.gibbins.
The following user(s) said Thank You: admin

Please Log in or Create an account to join the conversation.

More
1 year 1 month ago #25353 by admin
Replied by admin on topic Directory Monitor Issue
We will point multiple monitors to the same folder and check if we can reproduce the problem

Mike
ETL Architect

Please Log in or Create an account to join the conversation.

More
1 year 1 month ago - 1 year 1 month ago #25355 by bruce.gibbins
Thanks Mike. We have had it in this shape for at least 12 months prior to recent batch of upgrades. 

I have been able to reproduce and the number of monitors pointing to the same folder does not seem to be a factor. It appears to be an issue related to the fact that the folder being monitored is on a network share. If my test sample folder is a local drive folder then only a single queue entry is created as expected. Also the directory monitor shows the file being added but not the extra 'modification' entries

The last sample log I have loaded shows the result of monitoring a folder share versus a local drive folder

 




 
Last edit: 1 year 1 month ago by bruce.gibbins.
The following user(s) said Thank You: Peter.Jonson

Please Log in or Create an account to join the conversation.

Cookies user preferences
We use cookies to ensure you to get the best experience on our website. If you decline the use of cookies, this website may not function as expected.
Accept all
Decline all
Read more
Marketing
Set of techniques which have for object the commercial strategy and in particular the market study.
Google
Accept
Decline
Analytics
Tools used to analyze the data to measure the effectiveness of a website and to understand how it works.
Google Analytics
Accept
Decline
Google Analytics
Accept
Decline
Functional
Tools used to give you more features when navigating on the website, this can include social sharing.
Advertisement
If you accept, the ads on the page will be adapted to your preferences.
Google Ad
Accept
Decline
Save