File Name Grouping

  • Posts: 368
  • Karma: 2
  • Thank you received: 32

File Name Grouping was created by bruce.gibbins

Hi,

I don't think I have seen anything that could be used to do this. But we often have a set of files all in the same directory that have some type of grouping. For example,

file-20201230-A.csv
file-20201230-B.csv
file-20201230-A.pdf
file-20201231-A.csv
file-20201231-C.csv
file-20201231-A.pdf

The may arrive at the same time or we may be just processing a historic batch of data. But in this case the YYYYMMDD portion of the file needs to be used to group the files into a single batch when using a File Operation.

The YYYYMMDD portion in this case is a Date, but it has not relevance or relationship with the current system date. it also could be some other set of characters like a unique identifier.

My suggestion, is that we would find it useful if the File Operation List action could be extended to look for a REGEX and return each unique GROUP it finds, So in this case 20201230 and 20201231 then we could use the list it returns in a loop to then grab each group of files and process them as a group (eg. ZIP them up) for later downstream processing.

cheers
3 months 3 weeks ago #20486

Please Log in or Create an account to join the conversation.

  • Posts: 1140
  • Karma: 3
  • Thank you received: 136

Replied by Peter.Jonson on topic File Name Grouping

I think we need to review how the mask works everywhere and add check box "Use regular expression" where appropriate.

Some of our customers reported issues after we introduced regular expressions to email action.

www.etl-tools.com/forum/advanced-etl-pro...ail-error.html#20480
Peter Jonson
Support Analist
3 months 3 weeks ago #20489

Please Log in or Create an account to join the conversation.

 

This site uses cookies. By continuing to browse the site, you are agreeing to our use of cookies