- Posts: 539
- Thank you received: 59
AETL RegEx expression from Package Variable
- bruce.gibbins
- Topic Author
- Offline
- Platinum Member
-
Less More
9 months 2 weeks ago - 9 months 2 weeks ago #21978 by bruce.gibbins
AETL RegEx expression from Package Variable was created by bruce.gibbins
AETLE(P) v6.3.10.x (x32 | x64)
Hi I was wondering if you would consider or see value in being able to pass in an Regular Expression via a Package Variable into a Transformation Action?
My use case is that I'd like to have a reusable way of getting a set of files in a directory that could be grouped by a portion of their filenames and then treated as individual groups of files.
These file groups can then be zipped up together and then moved through the data flow as a distinct group/batch and then handled downstream separately from each other.
For example and this is just a simulation as the file names will be more complicated with dates in them etc
Files in directory could be
FILE_A_DATE1.csv
FILE_B_DATE1.pdf
FILE_A_DATE2.csv
FILE_B_DATE2.pdf
The end result would be the *DATE1* files group and the *DATE2* files as another group.
The idea is that I have a standalone transformation with a File System reader that is given the Root Folder from a Connection Object.
A list of files in that folder is read and passed to a Transform Action that generates a Group of values based on a RegEx expression that is collected from a GetPackage Action.
The results that are written to a local text file that can then be used in a downstream Loop Package Action.
Something like the images attached.
The whole idea is that we can just keep reusing this transformation with other packages and change the input via Package Variables and Connection Variables per parent package.
Hi I was wondering if you would consider or see value in being able to pass in an Regular Expression via a Package Variable into a Transformation Action?
My use case is that I'd like to have a reusable way of getting a set of files in a directory that could be grouped by a portion of their filenames and then treated as individual groups of files.
These file groups can then be zipped up together and then moved through the data flow as a distinct group/batch and then handled downstream separately from each other.
For example and this is just a simulation as the file names will be more complicated with dates in them etc
Files in directory could be
FILE_A_DATE1.csv
FILE_B_DATE1.pdf
FILE_A_DATE2.csv
FILE_B_DATE2.pdf
The end result would be the *DATE1* files group and the *DATE2* files as another group.
The idea is that I have a standalone transformation with a File System reader that is given the Root Folder from a Connection Object.
A list of files in that folder is read and passed to a Transform Action that generates a Group of values based on a RegEx expression that is collected from a GetPackage Action.
The results that are written to a local text file that can then be used in a downstream Loop Package Action.
Something like the images attached.
The whole idea is that we can just keep reusing this transformation with other packages and change the input via Package Variables and Connection Variables per parent package.
Last edit: 9 months 2 weeks ago by bruce.gibbins.
Please Log in or Create an account to join the conversation.
- Peter.Jonson
-
- Offline
- Platinum Member
-
9 months 2 weeks ago #21980 by Peter.Jonson
Peter Jonson
ETL Developer
Replied by Peter.Jonson on topic AETL RegEx expression from Package Variable
We will have a look
Peter Jonson
ETL Developer
The following user(s) said Thank You: bruce.gibbins
Please Log in or Create an account to join the conversation.
- Peter.Jonson
-
- Offline
- Platinum Member
-
9 months 3 days ago #21996 by Peter.Jonson
Peter Jonson
ETL Developer
Replied by Peter.Jonson on topic AETL RegEx expression from Package Variable
Peter Jonson
ETL Developer
The following user(s) said Thank You: bruce.gibbins
Please Log in or Create an account to join the conversation.