Data Warehousing and Data integration

ETL stops responding when reading large xml files

More
5 years 1 month ago #21265 by Peter.Jonson
I did some digging and found a way to speed things up.
I will let you know once an update is available

Peter Jonson
ETL Developer

Please Log in or Create an account to join the conversation.

More
5 years 1 month ago #21266 by dickels112
Hey Mike,

Thanks for the swift response. Are you maybe available for a short teams call?
I have some questions about (seems to be basic) functionalities that i cannot find in the documentation.

Thanks!

KR

Please Log in or Create an account to join the conversation.

More
5 years 1 month ago #21267 by Maria
Please post your questions here so everyone will benefit from the answers.

Start a new topic if it is a new issue

Please Log in or Create an account to join the conversation.

More
5 years 1 month ago #21272 by Peter.Jonson
The new release is ready.

Changes are.

23/07/2021 - 6.3.7.27

+ Improved: XML Reading performance
- Fixed: XSLT Transformation Action variable inside file name bugs

It turns out that the XML components we used had too much "overhead code"
After spending time optimising the code we have managed to make it make work much faster.
It takes around two minutes to process your file. (I think converting it to CSV first is a much better option in any case)



Thank you for pointing out this issue.
Any problems let us know

Peter Jonson
ETL Developer
Attachments:
The following user(s) said Thank You: bruce.gibbins

Please Log in or Create an account to join the conversation.

Cookies user preferences
We use cookies to ensure you to get the best experience on our website. If you decline the use of cookies, this website may not function as expected.
Accept all
Decline all
Read more
Marketing
Set of techniques which have for object the commercial strategy and in particular the market study.
Google
Accept
Decline
Analytics
Tools used to analyze the data to measure the effectiveness of a website and to understand how it works.
Google Analytics
Accept
Decline
Google Analytics
Accept
Decline
Functional
Tools used to give you more features when navigating on the website, this can include social sharing.
Advertisement
If you accept, the ads on the page will be adapted to your preferences.
Google Ad
Accept
Decline
Save