Avanced ETL Processor Pivot Table vs SQL Pivot Table

More
5 years 2 months ago - 5 years 2 months ago #18239 by DeanCovey
This is running against a very large SQL table. However, you can see the SQL version of the pivot table is much faster. Over an hour vs 30 seconds.








Just thought I would share.
Attachments:
Last edit: 5 years 2 months ago by DeanCovey.

Please Log in or Create an account to join the conversation.

More
5 years 2 months ago #18245 by DeanCovey
Just to clarify, I do updates and inserts:

Attachments:

Please Log in or Create an account to join the conversation.

More
5 years 2 months ago #18255 by admin
Thank you for the feedback

We will have a look at it.

Mike
ETL Architect

Please Log in or Create an account to join the conversation.

More
5 years 2 months ago #18292 by Peter.Jonson
Hi Dean

We have made some minor tweaks to Pivot and UnPivot objects and we hope that it works a little bit faster now.

Full list of chages

+ Added: GetWindowsTemporaryDirectory,GetTemporaryFile,GetTemporaryFileInDirectory,GetTemporaryDirectory transformation language functions
+ Added: Get File Content Dynamic transformation function
+ Improved: Third party components update
+ Improved: Pivot and unpivot performance

Peter Jonson
ETL Developer
The following user(s) said Thank You: DeanCovey

Please Log in or Create an account to join the conversation.

More
5 years 2 months ago #18324 by DeanCovey
I will definitely have to check into it. I checked this morning. For some reason, it's back to take an hour again.


Attachments:

Please Log in or Create an account to join the conversation.

More
5 years 2 months ago #18326 by DeanCovey
I was able to get my time back. Found out. the mappings have all disappeared. Added all the mappings back and life was good again.


Attachments:

Please Log in or Create an account to join the conversation.