Delimited File to Parquet Dynamic Action

About

Apache Parquet is an open-source, columnar storage file format designed for efficient data processing and analytics. It is optimized for use with big data frameworks such as Apache Hadoop, Apache Spark, and Apache Hive.

Unlike row-based formats (such as CSV or JSON), Parquet stores data by columns rather than by rows. This columnar layout enables high compression rates and faster query performance, particularly for analytical workloads that read only a subset of columns. Parquet supports efficient encoding schemes, nested data structures, and built-in schema definition, making it well suited for complex and large-scale datasets.

Because of its performance, compression efficiency, and strong ecosystem support, Parquet has become a standard format for data lakes, ETL pipelines, and business intelligence applications.

Metadata tab

1. The Write Variables option ensures that the “Package Action” logs the variable values into the “Action vars before log file” before the action is executed and logs the updated variable values into the “Action vars after log file” after the action completes.
2. This option is typically used for debugging and is not recommended in a production environment.
3. It can be globally disabled through the options dialog .

Parameters tab

Variables tab

The Variables tab defines the variables to be updated during the action’s execution.

Execution log tab

The Execution log tab is populated after the action is executed. Double click on the row to view the log.

Useful Links

• Debugging Actions
• Working With Variables

Video Tutorial