First of all, what is data?

Data can be any sort of information stored in a file or in a database

What does working with data means?

Loading data into the database, exporting data from the database,  transforming data from one format into another and validating

Why automating working with data is important?

The Data may be stored on the FTP server or in the Cloud. Coping this data manually can be a very time consuming and costly operation. Our software can connect to remote servers and download the data automatically. Here is a basic automation workflow example.  The boxes are called action. Every action is designed to do one job well. The action can have two statuses success (blue) or failure (orange). The green box is a starting point. In this example, the data is received via email. 

Advanced ETL Processor Sending Price List

Executing workflows

The workflows can be executed on regular basis using a scheduler or they may be started when a certain event happens. For example, the user creates a file in the directory or a new file is uploaded into the FTP server.

Supported data files

Text, XML, Excel, Hyper, TDE, JSON, HTML, QVD, QVX 

Supported databases

MS SQL Server, Oracle, MySql, MariaDB, PostgreSQL, Greenplum, Firebird, Interbase, SQLite, Access, DBF, Foxpro, Clarion, Paradox (BDE), any ODBC/OLE DB  compliant database, Google Spreadsheets, SalesForce, Brightpearl

Additional connections

POP3, IMAP4, SMTP, HL7, HTTP, File System, FTP, Cloud storage, Azure, Amazon, RSS, Windows Event Log, Active directory 

Here are some examples of what our customers use our ETL software for:
Extracting data from:
  • Multiple Delimited or Fixed width Text files
  • Multiple Excel files + Multiple Excel Spreadsheets
  • Multiple MS Access Databases + Multiple Tables
  • XML Files
  • Qvx and Qvd QlikView files
  • Multiple DBF Files
  • Multiple Clarion DAT Files
  • Any JDBC compliant database
  • Full support for OLE DB
  • Direct connection to Oracle, SQL Server, MySql, PostgreSQL, SQLite, SQL Server Compact and Interbase/Firebird
  • Multiple Tables
  • File System
  • POP3 Servers
  • IMAP4 Servers
  • Windows Event Log
  • TCP/IP Socket
  • RSS
  • Google SpreadSheet
  • SalesForce
  • Brightpearl
  • Slemma
  • BDE
  • Tableau Hyper Files
And Loading it into:
  • Multiple Data Targets
  • SQL scripts execution before and after loading
  • Delimited or Fixed width Text files
  • MS Access Database
  • Microsoft Excel File
  • DBF File
  • Any ODBC compliant database
  • Any OLE DB compliant database
  • Oracle database
  • Microsoft SQL Server database
  • Microsoft SQL Server CE
  • SQLite
  • MySQL
  • PostgreSQL
  • Interbase/Firebird
  • File System
  • SMTP Servers
  • XML Files
  • QlikView Qvx and Qvd files
  • SalesForce
  • Slemma
  • Google SpreadSheet
  • BDE
  • Tableau Hyper File
  • Tableau TDE File
Validation:
  • Deal With Dirty Data
  • Validate Postcodes, Phone numbers, TAX Codes using regular expressions
  • Validate Date Formats
  • Validate Date and Time
  • Validate Numbers
  • Validate Strings
Transformation:
  • Split, merge Fields
  • Split data and load it into different tables
  • Add, replace, delete strings
  • Run custom transformations
  • Translate coded values
  • Derive a newly calculated value
  • Sort, group and deduplicate data
Automation:
  • Load Data into ANY database
  • Export Data from ANY database
  • FTP Downloads and Uploads + SSL
  • Create or Delete directories on the FTP server
  • Rename or delete files on the FTP server
  • Send Emails with attachments
  • Receive Emails with attachments
  • Save attachments
  • Compress files
  • Decompress files
  • Copy, move, rename, merge, delete files
  • Compare files using MD5, size or creation date
  • Check if the file exists
  • Check Disk space
  • Generate Reports
  • Execute Different part of the package depending on the day of the week
  • Schedule packages for execution
  • Run bat files or execute any windows application
  • Edit and run SQL Scripts
  • Execute Packages from the command line
  • Add Records
  • Update Records
  • Delete Records
  • Add New and Update existing Records
  • De-duplicate Data
  • Sort Data
  • Group Data
  • Summarize Data
  • Pivot and UnPivot Data
  • Create surrogate keys
  • Run SQL script before and after loading data
  • Filter Source Data
  • Use Select Statement as a Data Source
  • Process Unicode Data
  • Generate XML
  • Generate SQL Insert Script Using any Table or File
  • Process EMAILS
  • Send EMAILS
  • Send SMS Messages
  • HTTP Downloads
  • Post HTTP Forms
  • Post Tweets
  • Ping servers
  • Check database state
  • Load images into a database
  • Export blobs into separate files
500 Transformation and Validation functions