[Req] Making Loop easier to resume/restart

More
1 year 1 month ago #25301 by prashant
Hellos,

I love loops & please would like to propose on few changes to make it easier for new users to work with them. 

Currently user has to do a lot of work in getting the LOOP resume to work to create loop counter file , check it's position , then check after loop starts etc

I had like to propose to integrate them all inside the LOOP counter itself with additional variables , to remove all the burden from user , AETL anyways excels at removing complexities and providing users with simple lego blocks to make a strong package.

1. We don't ask the user the position of the loop file or its location , we simply keep it in logs folder directly OR in the log running 
2. If above it not feasible , then we can ask the user for it's loop counter file location but default will be better 
3. Assuming the loop runs first time , no counter is found and we simple run the item .
4. Before running loop we store the sum number of rows for loop from sql or csv file , we then compare the log counter against it , 
4. When we run the loop again upon failure , we could show a message (again controlled by error message flag (which is turnen on by default) , last loop ran 123/345 , resume or restart loop
5. this will enhance the new loop dynamic package immensely in line
6. This should be made available for all existing loops , the default behaviour is always YES to restart/or ask user , since user won't miss this feature ,until they do have failure and then it would be too late

Cheers,
PM
The following user(s) said Thank You: Peter.Jonson

Please Log in or Create an account to join the conversation.

More
1 year 1 month ago - 1 year 1 month ago #25303 by Peter.Jonson
Hi PM

Thank you for you feed back.

I think what we need to do is to add two a parameter to loop actions

1 Resume Loop
2 Loop Position File

When Resume Loop is true current loop position value is saved into Loop Position File automatically
The same file is used to resume file execution
Upon successful completion the Loop Position File is deleted
You ca use Check File and Ask Question action from user interaction

I think it is a reasonable compromise,
We do not want add "Ask Question" functionality to loop actions.

We have a separate action for it.



 
 

Peter Jonson
ETL Developer
Last edit: 1 year 1 month ago by Peter.Jonson.

Please Log in or Create an account to join the conversation.

More
1 year 1 month ago #25308 by prashant

When Resume Loop is true current loop position value is saved into Loop Position File automatically
The same file is used to resume file execution
Upon successful completion the Loop Position File is deleted
You ca use Check File and Ask Question action from user interaction

I think it is a reasonable compromise

FAIR ENOUGH , this is reasonable.

1. Question is should resume loop value be on by default from settings ?
2. When the loop file is not deleted, then what happens ? By default we resume ? or we again overwrite the same ? or we need to apply for check file position to resume ? 
The following user(s) said Thank You: admin

Please Log in or Create an account to join the conversation.

More
1 year 1 month ago #25310 by admin
Some good news we have implemented your suggestion.
(Unlike working wit MS Teams it is a much easier thing to do)

19/06/2025 6.4.5.62

 + Added: "Resume Loop" and "Loop Position File" Parameters to Loop Dynamic Actions



Regarding your questions

1. Question is should resume loop value be on by default from settings ?

>>> It is false by default

2. When the loop file is not deleted, then what happens ? By default we resume ? or we again overwrite the same ? or we need to apply for check file position to resume ? 

>>> Loop Position File is deleted automatically upon successful completion of the loop.
>>> If Loop Position File exists and Resume Loop=true, Loop will resume automatically

We will add an example to the default repository later
 

Mike
ETL Architect
The following user(s) said Thank You: prashant

Please Log in or Create an account to join the conversation.

More
1 year 1 month ago #25314 by prashant
Will test this , hoping no bugs during deletion happens !
The following user(s) said Thank You: admin

Please Log in or Create an account to join the conversation.

More
1 year 1 month ago #25319 by prashant
latest version when I donwloda is .61

Please Log in or Create an account to join the conversation.

Cookies user preferences
We use cookies to ensure you to get the best experience on our website. If you decline the use of cookies, this website may not function as expected.
Accept all
Decline all
Read more
Marketing
Set of techniques which have for object the commercial strategy and in particular the market study.
Google
Accept
Decline
Analytics
Tools used to analyze the data to measure the effectiveness of a website and to understand how it works.
Google Analytics
Accept
Decline
Google Analytics
Accept
Decline
Functional
Tools used to give you more features when navigating on the website, this can include social sharing.
Advertisement
If you accept, the ads on the page will be adapted to your preferences.
Google Ad
Accept
Decline
Save