Allow subdirectories in Reader File Mask

More
3 years 7 months ago #19675 by bruce.gibbins
Hi. This may be something we are doing wrong or it may be an enhancement.

It seems that when you are setting a reader connection, you can't then extend the initial path that the connection provides to include a sub-directory. There is a 'search in sub-directories' option but in our case it would be more convenient and possibly less overhead on the server if we could specify a sub-directory as part of the file mask that is then combined with the Connection path.

The screen shots show that we have included a sub-directory in the Mask field along with a wildcard mask for the files required. In the log the Reader indicates it has found the files but then when it performs the actual read it doesn't use the true path to the files but seems to take Directory Connection Path and the actual filename itself and ignores the sub-directory we put in to the mask. We are doing this to try make things as dynamic as we can with fewer hard-coded directory connections. What we would like to see is that the sub-directory portion of the Mask can be a variable.

The idea is that the connection path points to somewhere like C:\folderA

Then we do some package magic and drop a file into C:\folderA\UNIQUEID\ Where UNIQUEID is a generated number. This way we can avoid clashes on the server in FolderA where there is the chance files of the same name may come in simultaneously fro different sources and all land in FolderA at the same time. UNIQUEID which would be a variable (typically a timestamp) can then be used to hold a number of files in a self-contained fashion. Since it is dynamically generated we can then just have a single Directory Connection that can be reused elsewhere with or without sub-directories if need be.


Attachments:

Please Log in or Create an account to join the conversation.

More
3 years 7 months ago #19676 by admin
There is a much easier option which you can use right now.
Just place variable inside the directory path.


Mike
ETL Architect
Attachments:
The following user(s) said Thank You: bruce.gibbins

Please Log in or Create an account to join the conversation.

More
3 years 7 months ago #19678 by bruce.gibbins
Yes, thanks. I had realised that and we use it extensively in our ecosystem. However, it just so happens in this case I wanted to use a connection that is already made up of variables and I didn't want to alter it and possibly cause issues with existing objects that use it. So I have just created another connection based on the original one. It is a minor thing as I wanted to avoid creating yet another connection that may make things confusing in the future.

But thanks all the same
The following user(s) said Thank You: Peter.Jonson

Please Log in or Create an account to join the conversation.