- Posts: 5
- Thank you received: 0
Capture data with the same value from multiple source file and extract to new DB
7 years 8 months ago #14322
by Jan
Need help on this.
I have several source file with similar headers, all I want to do is to export those files that have similar values and count the numbers. All data that does not have a value will be omitted from the exported file.
Example:
FILE 1
Name1,98
Name2,99
Name3,94
Name5,90
FILE 2
Name1,98
Name4,97
Name3,94
Name7,88
FILE 3
Name2,99
Name3,94
Name1,98
Name6,90
RESULT should be:
NAME1,98,3
NAME2,99,2
NAME3,94,3
Thanks!
I have several source file with similar headers, all I want to do is to export those files that have similar values and count the numbers. All data that does not have a value will be omitted from the exported file.
Example:
FILE 1
Name1,98
Name2,99
Name3,94
Name5,90
FILE 2
Name1,98
Name4,97
Name3,94
Name7,88
FILE 3
Name2,99
Name3,94
Name1,98
Name6,90
RESULT should be:
NAME1,98,3
NAME2,99,2
NAME3,94,3
Thanks!
Please Log in or Create an account to join the conversation.
7 years 8 months ago #14323
by Neale
Replied by Neale on topic Capture data with the same value from multiple source file and extract to new DB
If your files are CSVs and you don't mind some pre-processing - Google CSVFIX - Very useful command line utility
Please Log in or Create an account to join the conversation.
7 years 8 months ago #14329
by admin
Mike
ETL Architect
Replied by admin on topic Capture data with the same value from multiple source file and extract to new DB
Jan, thank you for the question.
What you need to do is to sort the data first,
group it and count number of values.
Than filter out those records where number of values = 1
The desired result
What you need to do is to sort the data first,
group it and count number of values.
Than filter out those records where number of values = 1
The desired result

Mike
ETL Architect
Please Log in or Create an account to join the conversation.
7 years 8 months ago #14330
by admin
Mike
ETL Architect
Replied by admin on topic Capture data with the same value from multiple source file and extract to new DB
Actual transformation is attached, extract all files into c:\temp folder and open sample.ats file
Mike
ETL Architect
The following user(s) said Thank You: Jan
Please Log in or Create an account to join the conversation.
7 years 7 months ago #14350
by Jan
Replied by Jan on topic Capture data with the same value from multiple source file and extract to new DB
Many thanks Mike! This helps a lot.

Please Log in or Create an account to join the conversation.