Advanced ETL - Transformation Editor Set Package Variable

More
1 month 1 week ago #26133 by hsdaybright
How do I properly use the "Set Package Variable" action in the transformation editor?
A simple example: I've set up a sequence to count all the rows in the source and place that in a #COUNT# variable.
If I use the "Set Variable" action, it works perfectly, but I can't retrieve the variable in another package action. But, using the "Set Package Variable" slows down the processing by a lot.
Is there a way to set a transformation action to only run once at the end of the transformation? Or maybe use a script after the transformer and before the writer to set a global package variable from the local variable?
 

Also, what is the "Count Number of Records" option for Oracle and Sql sources? Does it store that in a variable I can access? This won't solve all the counting requirements I have as I will do counts by groups and calculations as well, but it can help with a simple "count all records" requirement.
 
The following user(s) said Thank You: admin

Please Log in or Create an account to join the conversation.

More
1 month 1 week ago - 1 month 1 week ago #26134 by admin
Regarding "Count Number of Records"
When you run a transformation, Advanced ETL Processor displays a progress bar and completion percentage.

To calculate this percentage, the software must first determine the total number of records in the source table.This record-counting operation can take a significant amount of time, especially for large tables, and may negatively impact overall performance.

Disabling Count Number of Records prevents the software from counting the records before processing begins, allowing the transformation to start immediately and potentially improving performance.
 

Mike
ETL Architect
Last edit: 1 month 1 week ago by admin.

Please Log in or Create an account to join the conversation.

More
1 month 1 week ago - 1 month 1 week ago #26135 by admin
Set Package Variable works with all data sources, not just SQL databases.At the moment, there is no "Set Once at the End" option available. The variable is updated each time the action is executed during processing. 




Mike
ETL Architect
Last edit: 1 month 1 week ago by admin.

Please Log in or Create an account to join the conversation.

More
1 month 1 week ago - 1 month 2 days ago #26136 by admin
Since you are working with SQL server there is more options available

www.etl-tools.com/wiki/advanced-etl-proc...to-variables-action/

or

www.etl-tools.com/wiki/advanced-etl-proc...to-variables-action/



Any more questions let me know 

Mike
ETL Architect
Last edit: 1 month 2 days ago by admin.

Please Log in or Create an account to join the conversation.

Cookies user preferences
We use cookies to ensure you to get the best experience on our website. If you decline the use of cookies, this website may not function as expected.
Accept all
Decline all
Read more
Marketing
Set of techniques which have for object the commercial strategy and in particular the market study.
Google
Accept
Decline
Analytics
Tools used to analyze the data to measure the effectiveness of a website and to understand how it works.
Google Analytics
Accept
Decline
Google Analytics
Accept
Decline
Functional
Tools used to give you more features when navigating on the website, this can include social sharing.
Advertisement
If you accept, the ads on the page will be adapted to your preferences.
Google Ad
Accept
Decline
Save