Advanced ETL Processor can simplify distinct string aggregation with the Grouper transformation and the `Deduplicate Delimited Values` string function. Use `Concatenate Unique` when grouping rows, and use the string function when one field already contains repeated delimited values.
What You Will Learn
- Group records by one or more key fields.
- Concatenate unique string values inside each group.
- Deduplicate values inside an existing delimited string.
- Decide which fields should pass to the next transformation.
- Sort or deduplicate data when the upstream layout needs it.
Before You Start
Use sample rows with deliberate duplicate values and at least one field that already contains a repeated delimited list. If every input value is already unique, the test proves very little and still manages to look confident.
Distinct String Docs
The Deduplicate Delimited Values documentation covers the string function used when one field already contains repeated values separated by a delimiter.
Use that function in the Transformer when the input looks like `A,B,A,C` and the output should keep each value once. Double click the transformation function to review its properties before running it against live data.
When the duplicate values come from multiple rows instead, use the Grouper. Set the key columns to `Group By` and the text column to `Concatenate Unique` when each group needs one distinct list.
Grouper does not sort data, so use Sorter first when output order matters. If the source has duplicate records rather than just duplicate string values, review the Deduplicator before grouping.
Need Advanced ETL Processor First?
Download the Advanced ETL Processor trial before following the tutorial, or review Enterprise if you need scheduled unattended execution.