Looping Around a List of Files to Perform a Transformation of the File Data

More
4 years 8 months ago #21378 by ipitzey
Hi,

I have setup a package to obtain a list of files and supply this to the Loop action, which works.
I'm trying to loop round each file to transform some of the fields within the file, to remove problematic data etc.
If I connect a Transform action to the loop as shown, I can't get the Transform to read each file and perform the transformation on the delimited file.





Can anyone advise, what I need to do, or what I need to change to get this to work?

Many thanks,
Ian
Attachments:

Please Log in or Create an account to join the conversation.

More
4 years 8 months ago #21379 by ipitzey
As an update, it looks like the Transformer Reader object is not recognising files in the directories. Even if I create a new Transform Object and try to connect the Reader to the Directory, which is fine, it then just doesn't find the files?????? I must be doing something wrong, the files are local on the C:\ and is not a mapped drive?

Please Log in or Create an account to join the conversation.

More
4 years 8 months ago #21380 by admin

Please Log in or Create an account to join the conversation.

More
4 years 8 months ago - 4 years 8 months ago #21381 by Peter.Jonson
Hi Ian.

I have found your post very confusing...
Our software is capable of loading the data from multiple files using mask.
Yes, it is possible to loop through the files list but it makes things unnecessarily complicated.
Perhaps I am missing something...

Peter Jonson
ETL Developer
Attachments:
Last edit: 4 years 8 months ago by Peter.Jonson.

Please Log in or Create an account to join the conversation.

More
4 years 8 months ago #21382 by ipitzey
Hi Mike,

Thanks for the reply.I have strimmed the package down to not loop and just use the Transformation object, to loop through the directory. I still get the error "Writer: {mywriter} Nothing is Mapped as your video shows, but both running it as the Package, as well as the individual object. My Source and Destination directories are setup correctly and the Source Directory contains the files, whilst the Destination Directory contains an the correct directory, with no files currently within. I have attached some screen shots of what's setup. I cannot see what is wrong?
Thanks,
Ian





Please Log in or Create an account to join the conversation.

More
4 years 8 months ago #21383 by ipitzey
Thanks Peter,

Please see the message above.

Thanks for helping.
Ian

Please Log in or Create an account to join the conversation.

Cookies user preferences
We use cookies to ensure you to get the best experience on our website. If you decline the use of cookies, this website may not function as expected.
Accept all
Decline all
Read more
Marketing
Set of techniques which have for object the commercial strategy and in particular the market study.
Google
Accept
Decline
Analytics
Tools used to analyze the data to measure the effectiveness of a website and to understand how it works.
Google Analytics
Accept
Decline
Google Analytics
Accept
Decline
Functional
Tools used to give you more features when navigating on the website, this can include social sharing.
Advertisement
If you accept, the ads on the page will be adapted to your preferences.
Google Ad
Accept
Decline
Save