- Posts: 539
- Thank you received: 59
FTP Operation Option
- bruce.gibbins
- Topic Author
- Offline
- Platinum Member
-
Less More
1 year 6 months ago #21432 by bruce.gibbins
FTP Operation Option was created by bruce.gibbins
AETLE/P v6.3.7.xx
Hi,
I was wondering if it others may find this useful?
If an internal retry option was added to the FTP Operation Action which basically provided some type of limited retry loop if an error occurred during the event then this would avoid having to manufacture a loop in the parent package and reduce the 'noise' in teh package layout.
For example, we have a scenario where we pickup files from an external FTP host but on occasion they may be dumping files in at the same time or the host hasn't released the locks on the file and we get an error such as the following.
In most cases the next pickup will run 15 minutes later and collect the file mentioned in the error. For us the time these files come in is not critical. But if we could avoid generating the error by having a simple (and limited) internal retry loop then I can see this helping us.
Kind Regards
Bruce
Hi,
I was wondering if it others may find this useful?
If an internal retry option was added to the FTP Operation Action which basically provided some type of limited retry loop if an error occurred during the event then this would avoid having to manufacture a loop in the parent package and reduce the 'noise' in teh package layout.
For example, we have a scenario where we pickup files from an external FTP host but on occasion they may be dumping files in at the same time or the host hasn't released the locks on the file and we get an error such as the following.
Code:
Error happened during downloading fileName.csv SFTP server error occurred: Permission denied.
In most cases the next pickup will run 15 minutes later and collect the file mentioned in the error. For us the time these files come in is not critical. But if we could avoid generating the error by having a simple (and limited) internal retry loop then I can see this helping us.
Kind Regards
Bruce
Please Log in or Create an account to join the conversation.
1 year 6 months ago #21433 by admin
Mike
ETL Architect
Replied by admin on topic FTP Operation Option
You have a 15-minute window to process the files. if we introduce "unlimited retry", one day your execution will get stuck and nothing will be processed until you manually resolve the issue.
There is an automatic retry for FTP/FTPES already but not for SFTP.
We will have a look at adding automatic retry to SFTP for you
Our software makes 3 attempts to download/upload the files and it fails after the third attempt.
There is no guarantee that it is sufficient for your supplier to finish uploading the files.
There is an automatic retry for FTP/FTPES already but not for SFTP.
We will have a look at adding automatic retry to SFTP for you
Our software makes 3 attempts to download/upload the files and it fails after the third attempt.
There is no guarantee that it is sufficient for your supplier to finish uploading the files.
Mike
ETL Architect
Please Log in or Create an account to join the conversation.
- bruce.gibbins
- Topic Author
- Offline
- Platinum Member
-
Less More
- Posts: 539
- Thank you received: 59
1 year 6 months ago #21434 by bruce.gibbins
Replied by bruce.gibbins on topic FTP Operation Option
Thanks Mike.
My apologies the use of FTP was incorrect in all cases for us it is only ever SFTP.
You are 100% correct and I was not suggesting unlimited retries for exactly the reason you pointed out.
Three automatic retries for SFTP would be what I was after and if it happened with no configuration required then all the better. We have other procedures in play. So if the 3 retries are exceeded then we would just pickup the the next time the scheduled task runs.
Regards
Bruce
My apologies the use of FTP was incorrect in all cases for us it is only ever SFTP.
You are 100% correct and I was not suggesting unlimited retries for exactly the reason you pointed out.
Three automatic retries for SFTP would be what I was after and if it happened with no configuration required then all the better. We have other procedures in play. So if the 3 retries are exceeded then we would just pickup the the next time the scheduled task runs.
Regards
Bruce
Please Log in or Create an account to join the conversation.
1 year 6 months ago #21435 by admin
Mike
ETL Architect
Replied by admin on topic FTP Operation Option
Some good news for you: we have implemented your request in the latest beta version
www.etl-tools.com/articles/use-microsoft...fice-365-emails.html
There were a lot of small changes related to email processing which I believe you use a lot.
We are planning to make an official release on 1st of December
Any issues let us know
www.etl-tools.com/articles/use-microsoft...fice-365-emails.html
There were a lot of small changes related to email processing which I believe you use a lot.
We are planning to make an official release on 1st of December
Any issues let us know
Mike
ETL Architect
The following user(s) said Thank You: bruce.gibbins
Please Log in or Create an account to join the conversation.
- bruce.gibbins
- Topic Author
- Offline
- Platinum Member
-
Less More
- Posts: 539
- Thank you received: 59
1 year 6 months ago #21438 by bruce.gibbins
Replied by bruce.gibbins on topic FTP Operation Option
Thanks Mike. Appreciate the notification about the new Beta. The release notes look exciting and yes we do need to process a lot of inbound files coming via email.
Please Log in or Create an account to join the conversation.