Capture data with the same value from multiple source file and extract to new DB

More
7 years 8 months ago #14322 by Jan
Need help on this.

I have several source file with similar headers, all I want to do is to export those files that have similar values and count the numbers. All data that does not have a value will be omitted from the exported file.

Example:

FILE 1

Name1,98
Name2,99
Name3,94
Name5,90

FILE 2
Name1,98
Name4,97
Name3,94
Name7,88

FILE 3
Name2,99
Name3,94
Name1,98
Name6,90

RESULT should be:

NAME1,98,3
NAME2,99,2
NAME3,94,3

Thanks!

Please Log in or Create an account to join the conversation.

More
7 years 8 months ago #14323 by Neale
If your files are CSVs and you don't mind some pre-processing - Google CSVFIX - Very useful command line utility

Please Log in or Create an account to join the conversation.

More
7 years 8 months ago #14329 by admin
Jan, thank you for the question.
What you need to do is to sort the data first,
group it and count number of values.
Than filter out those records where number of values = 1





The desired result :)

Mike
ETL Architect

Please Log in or Create an account to join the conversation.

More
7 years 8 months ago #14330 by admin
Actual transformation is attached, extract all files into c:\temp folder and open sample.ats file

Mike
ETL Architect
Attachments:
The following user(s) said Thank You: Jan

Please Log in or Create an account to join the conversation.

More
7 years 7 months ago #14350 by Jan
Many thanks Mike! This helps a lot. ;)

Please Log in or Create an account to join the conversation.