Advanced ETL Processor can simplify distinct string aggregation with the `Deduplicate Delimited Values` transformation function. Use it when one field already contains a list, such as `red,blue,red`, and the output should keep each value once.
What You Will Learn
- Find the string transformation function inside the Transformer.
- Remove duplicate values from an existing delimited field.
- Check the function properties before running the transformation.
- Decide when a string function is simpler than a grouping step.
Before You Start
Use a field with repeated delimited values, for example `red,blue,red`. If the duplicates are spread across separate rows, use Grouper instead. Different problem, different tool.
Delimited Value Docs
The Deduplicate Delimited Values documentation covers the string transformation function used when duplicate values are already stored inside one delimited field.
Use it in the Transformer when the source value looks like `A,B,A,C` and the cleaned value should keep each item once. Double click the function to review its properties before processing real files.
This is different from row-level aggregation. If duplicate values arrive as separate records, use the Grouper with `Concatenate Unique` instead.
Need Advanced ETL Processor First?
Download the Advanced ETL Processor trial before following the tutorial, or review Enterprise if you need scheduled unattended execution.