- Posts: 568
- Thank you received: 72
Directory Monitor
- bruce.gibbins
- Topic Author
- Offline
- Platinum Member
-
Less
More
2 years 9 months ago #20487
by bruce.gibbins
Directory Monitor was created by bruce.gibbins
Hi.
AETLE v6.3.7.7
We have a package that is meant to run when a set of files arrives into a specific folder (which happens to be an FTP Landing areas on our server).
in this case 3 files are placed there by the 3rd party, 2 CSV files and a PDF. I have built the package and it executes as expected. I then associated it with an Event Monitor which looks for the PDF file in the folder.
I noticed that 2 Entries appear in the execution log rather than 1 as expected.
The package is meant to see the PDF file (as there is only one) and then fire off the package which will then find all 3 files and moves them to another location for further downstream processing.
Why do 2 Execution Events appear in the Log? One actually fails and the other is successful.
Thanks
AETLE v6.3.7.7
We have a package that is meant to run when a set of files arrives into a specific folder (which happens to be an FTP Landing areas on our server).
in this case 3 files are placed there by the 3rd party, 2 CSV files and a PDF. I have built the package and it executes as expected. I then associated it with an Event Monitor which looks for the PDF file in the folder.
I noticed that 2 Entries appear in the execution log rather than 1 as expected.
The package is meant to see the PDF file (as there is only one) and then fire off the package which will then find all 3 files and moves them to another location for further downstream processing.
Why do 2 Execution Events appear in the Log? One actually fails and the other is successful.
Thanks
Please Log in or Create an account to join the conversation.
- Peter.Jonson
-
- Offline
- Platinum Member
-
2 years 9 months ago #20488
by Peter.Jonson
Peter Jonson
ETL Developer
Replied by Peter.Jonson on topic Directory Monitor
Some software creates an empty file first to check if the directory is writable or not.
An empty file is immediately deleted and the file is created again
This one of the possibilities.
Anyway, we will try to reproduce the problem in the office.
An empty file is immediately deleted and the file is created again
This one of the possibilities.
Anyway, we will try to reproduce the problem in the office.
Peter Jonson
ETL Developer
Please Log in or Create an account to join the conversation.
- Peter.Jonson
-
- Offline
- Platinum Member
-
2 years 9 months ago #20497
by Peter.Jonson
Peter Jonson
ETL Developer
Replied by Peter.Jonson on topic Directory Monitor
Hi Bruce
I have spent this morning trying to reproduce the issue.
I tried both 32 and 64-bit version of the monitor.
Every time it worked as expected eg only one entry was created
I used windows explorer to copy the files.
There is a way to write detailed file access information into the windows event log
www.lepide.com/how-to/track-changes-made...les-and-folders.html
That might tell us what is wrong.
I was not able to use the variable inside the file mask box
I have spent this morning trying to reproduce the issue.
I tried both 32 and 64-bit version of the monitor.
Every time it worked as expected eg only one entry was created
I used windows explorer to copy the files.
There is a way to write detailed file access information into the windows event log
www.lepide.com/how-to/track-changes-made...les-and-folders.html
That might tell us what is wrong.
I was not able to use the variable inside the file mask box
Peter Jonson
ETL Developer
Please Log in or Create an account to join the conversation.
- bruce.gibbins
- Topic Author
- Offline
- Platinum Member
-
Less
More
- Posts: 568
- Thank you received: 72
2 years 8 months ago #20508
by bruce.gibbins
Replied by bruce.gibbins on topic Directory Monitor
Hi.
I have retried this using the new AETLE v6.3.7.9 release and the issue I was seeing is NOT occurring. We will continue to monitor. I am wondering if there was an second orphaned AETLE Agent running.
regards
Bruce
I have retried this using the new AETLE v6.3.7.9 release and the issue I was seeing is NOT occurring. We will continue to monitor. I am wondering if there was an second orphaned AETLE Agent running.
regards
Bruce
Please Log in or Create an account to join the conversation.
- Peter.Jonson
-
- Offline
- Platinum Member
-
2 years 8 months ago #20512
by Peter.Jonson
Peter Jonson
ETL Developer
Replied by Peter.Jonson on topic Directory Monitor
Hi Bruce.
Thank you for the update.
"I am wondering if there was a second orphaned AETLE Agent running."
I doubt that.
Also, it is been a while since we modified directory monitir code
Thank you for the update.
"I am wondering if there was a second orphaned AETLE Agent running."
I doubt that.
Also, it is been a while since we modified directory monitir code
Peter Jonson
ETL Developer
Please Log in or Create an account to join the conversation.
- bruce.gibbins
- Topic Author
- Offline
- Platinum Member
-
Less
More
- Posts: 568
- Thank you received: 72
2 years 7 months ago #20719
by bruce.gibbins
Replied by bruce.gibbins on topic Directory Monitor
AETLE x64 v6.3.7.11
Hi
I can reproduce this at our end by doing the following
The Directory being monitored is on our network as a SHARE with a UNC Path. The path happens to be the home location for an inbound SFTP account holder
The screen shots show a mixture of real files coming into this home location from an external source and others which are me me copying/moving a file from another network location over to the same FTP home.
The external FTP client seems to be doing as you have suggested. Creating a TMP file and renaming it. My testing did not involve any TMP file but all of my tests as well as the LIVE inbound files from the remote source all generated 2 Execution Entries.
I have also included screenshots of the Monitor setup which is looking for "*unprocessed report*.csv"
If you notice, even if the tmp file did come in, it has an extension of .tmp and the pattern is for *.csv. so at the very least it still should have ignored the *.tmp until it became .csv
My copies using File Explorer did not seem to create any type of tmp file but still created 2 Execution entries. I was using WinSCP as an FTP client to send the file to the home folder and it did seem to create a .tmp file
So, to me it seems that the pattern matching on the monitor may be getting ignored or at least sending a false positive as I am explicitly looking for a "*.csv" file to hit the monitored folder.
BTW. As complete disclosure, the home folder is hosted on a Synology NAS which is also acting as the SFTP server.
Cheers
Hi
I can reproduce this at our end by doing the following
The Directory being monitored is on our network as a SHARE with a UNC Path. The path happens to be the home location for an inbound SFTP account holder
The screen shots show a mixture of real files coming into this home location from an external source and others which are me me copying/moving a file from another network location over to the same FTP home.
The external FTP client seems to be doing as you have suggested. Creating a TMP file and renaming it. My testing did not involve any TMP file but all of my tests as well as the LIVE inbound files from the remote source all generated 2 Execution Entries.
I have also included screenshots of the Monitor setup which is looking for "*unprocessed report*.csv"
If you notice, even if the tmp file did come in, it has an extension of .tmp and the pattern is for *.csv. so at the very least it still should have ignored the *.tmp until it became .csv
My copies using File Explorer did not seem to create any type of tmp file but still created 2 Execution entries. I was using WinSCP as an FTP client to send the file to the home folder and it did seem to create a .tmp file
So, to me it seems that the pattern matching on the monitor may be getting ignored or at least sending a false positive as I am explicitly looking for a "*.csv" file to hit the monitored folder.
BTW. As complete disclosure, the home folder is hosted on a Synology NAS which is also acting as the SFTP server.
Cheers
Please Log in or Create an account to join the conversation.