Here you will find the latest news and articles about our ETL Software

Our WIKI has all the information you need, if you are stuck, please post your question on our support forum and we will do our best to assist you

We are constantly working on optimizing the performance of our ETL software. Typical ETL process is very similar to a river. Narrow parts prevent water from flowing freely. But if there is no water widening narrows parts will not help. The data flow starts with the data reader. So it is important to make it very fast. It is so important to measure performance correctly.  We always use the same table structure and the same file format for testing.  We did spend a lot of time optimizing SQLite data reader and manage to make two times faster

Latest version Advanced ETL Processor extracts data from SQLite more than 2 times faster

etl sqlite 

Advanced ETL Processor Enterprise version 6.3.4.2 does 123,000 records per second and extracts 14mln records in 1 minute and 53 seconds

transform sqlite data up to two times faster

Advanced ETL Processor Enterprise version 6.3.4.6 does 364,000 records per second and extracts 14mln records in 38 seconds.

Download Free Trial Learn More About Our ETL Software

Starting from the middle of March 2019 users would need to enter the license code and control number to register the software.

Registration1

We used a very old licensing system and we wanted to change it for a very long time. License type can be now clearly seen on the about box and maintenance tab. This change applies to all our software products.

Advanced ETL Processor is true real alternative to SSIS. It's highly advanced capabilities make SSIS look like a toy.

VAST NUMBER OF CONNECTORS

Text, XML ,Excel, QVX, QVD, MS Access Database, DBF, Clarion, ODBC, JDBC, OleDB, MongoDB, MS SQL Server, MS SQL Server CE, Oracle, MySQL, PostgreSQL, Interbase/Firebird, SQLite, File System, POP3, IMAP4, HL7, Google SpreadSheet, RSS Feed, SalesForce, BrightPearl, BDE, HTTP, FTP, Microsoft Message Queue, Windows Event Log, Active Directory and JSON

Download Advanced ETL Processor Enterprise now!


Alternative to SSIS

GREAT PERFORMANCE - THOUSANDS OF RECORDS PER SECOND

etl performance

Performance depends on data and complexity of the transformation

TRANSFORM THE DATA - ANY WAY YOU WANT

Advanced ETL Processor has more than 500 transformation and validation functions. if something is missing we will add for you. Data can be sorted grouped or filtered, the possibilities are limitless. 


Transform Excel files anyway you want

WORKFLOW EXAMPLE

Automatic Invoice Processing

Workflow operations

all package automation actions

WE DO EVERYTHING TO MAKE LIFE OF DEVELOPERS EASIER

Here is SSIS Ftp task editor. There is no much to change or see here. Plus FTP is not secure protocol so it makes it useless in the corporate environment.

ssis ftp task

Here is Advanced ETL Processor FTP Action Properties dialogue. The users can see the files and perform necessary FTP actions.

ftp package action

UNIVERSAL DataReader AND WRITER

Unlike SSIS, Advanced ETL Processor avoids having different connectors, many of which are very subtle variations on similar components. Instead, Advanced ETL Processor uses only one universal Data writer and Data reader component but each one is highly configurable and includes all of the parameters needed to get your data in and out of almost any data source. This approach allows the end user to design mapping once and use it with any database or file.

MONITOR SERVICES

The monitor is windows service waiting for an event to happen. When it does it executes the predefined action. we offer the following

Monitor TypeDescription
Directory Monitor Tracks folder changes, such as file creation, removal, modification or rename.
FTP Monitor Tracks new files arrivals
IMAP4 Monitor Tracks new email arrivals
POP3 Monitor Tracks new email arrivals
MSMQ Monitor Checks Message Queue (MSMQ) for new messages.
HTTP Monitor a web server which executes an action when predefined URL is open.

 Interested?

Contact us now!

VIDEO TUTORIAL

FTP protocol is used daily by most businesses around the world. Our ETL Software makes it easy to automate FTP uploads and downloads.

COMPLETE FTP AUTOMATION

 The users can create and delete directories on the FTP server, delete files, move files and produce a list of files. Here is a basic example of FTP workflow. 

ftp automation workflow example

Download Advanced ETL Processor Enterprise now!

We also offer FTP monitor which constantly check FTP servers for new files arrivals. When a new file is detected it executes a predefined action.

ftp monitor 1

ftp monitor 2

ftp monitor 3

MAKING LIFE OF END USERS AND DEVELOPERS EASIER

We are constantly working on making our software better. This dialogue is fully functioning FTP client. It allows the users to see the files on the FTP server. But we did not stop there. The users can also download and upload files, create and delete remote directories. 

ftp package action

Supported FTP PROTOCOLS

  • FTP – File Transfer Protocol
  • FTPS – FTP over implicit TLS/SSL
  • FTPES – FTP over explicit TLS/SSL Require TLS
  • FTPES – FTP over explicit TLS/SSL
  • SFTP – SSH File Transfer Protocol

ftp connection dialogue

What is FTP

File Transfer Protocol (FTP) is a standard network protocol used to copy a file from one host to another over a TCP/IP-based network, such as the Internet. FTP is built on a client-server architecture and utilizes separate control and data connections between the client and server applications which solves the problem of different end host configurations (i.e. Operating System, file names). FTP is used with user-based password authentication or with anonymous user access.
FTP Automation with Visual  Importer ETL

How FTP works

A client makes a connection to the server on TCP port 21. This connection, called the control connection, remains open for the duration of the session, with a second connection, called the data connection, on port 20 opened as required to transfer file data. The control connection is used to send administrative data (i.e. commands, identification, passwords). Commands are sent by the client over the control connection in ASCII and terminated by a carriage return and line feed. For example "RETR filename" would transfer the specified file from the server to the client. Due to this two-port structure, FTP is considered out-of-band, as opposed to an in-band protocol such as HTTP.

The server responds on the control connection with three digit status codes in ASCII with an optional text message, for example, "200" (or "200 OK.") means that the last command was successful. The numbers represent the code number and the optional text represent explanations (i.e. <OK>) or needed parameters (i.e. <Need account for storing file>). A file transfer in progress over the data connection can be aborted using an interrupt message sent over the control connection.

FTP can be run in active mode or passive mode, which control how the second connection is opened. In active mode, the client sends the server the IP address port number that the client will use for the data connection, and the server opens the connection. Passive mode was devised for use where the client is behind a firewall and unable to accept incoming TCP connections. The server sends the client an IP address and port number and the client opens the connection to the server.

Security consideration

FTP has no encryption tools meaning all transmissions are in clear text; usernames, passwords, FTP commands and transferred files can be read by anyone sniffing on the network. This is a problem common to many Internet protocol specifications written prior to the creation of SSL, such as HTTP, SMTP and Telnet. The common solution to this problem is to use either SFTP (SSH File Transfer Protocol) or FTPS (FTP over SSL), which adds SSL or TLS encryption to FTP.

Anonymous FTP

A host that provides an FTP service may additionally provide anonymous FTP access. Users typically log in to the service with an 'anonymous' account when prompted for a username. Although users are commonly asked to send their email address in lieu of a password, no verification is actually performed on the supplied data.

Contact us now!

 

Good news we have just added support for cloud storage to our ETL products.

Supported Cloud Storage Operations: Download and Upload Files, Create and Delete Folders, List files.

Download Advanced ETL Processor Enterprise now!

Online storage systems, or “cloud” services as they're also known, allow you to store digital media online by uploading via your computer or mobile device. Though transferred and accessible via the web, the data is actually physically held in datacentres located around the world, often owned by whoever provides the particular service you use (Google, Microsoft or Dropbox).

Following clod storage providers are supported:

  1. DropBox
  2. Google Drive
  3. Microsoft OneDrive
  4. Box
  5. Amazon Cloud Drive
  6. Hubic
  7. HiDrive
  8. Yandex disk

Cloud Storage Action

The cloud storage action dialogue allows the end user to define package action properties. It also allows browsing , downloading and uploading files and works with directories.

Cloud Storage Action

This dialogue demonstrates a different attitude towards developers (end users). In fact, this is how development should be done. A lot of companies create MVP and nothing else. More than they are not willing to make it any better as long as management is happy to buy the software they are happy with the product. The problem is that Managers do not use the software I do.  The cloud storage dialogue make my life much easier

Tom Barker, 
Director of Information Technology

The "how to connect" guides can be found here

www.etl-tools.com/wiki/knowledgebase/start

Upgrade procedure

You would need to run the following SQL against the repository to enable cloud storage connection

Insert into OBJECT_TYPES
(OBJECT_TYPE,OBJECT_TYPE_NAME)
Values
('905','Cloud Storage Connection')

Insert into OBJECT_TYPES
(OBJECT_TYPE,OBJECT_TYPE_NAME)
Values
('906','Cloud Storage Connections')

INSERT INTO OBJECTS_TREE
(OBJECT_ID,
PARENT_ID,
NAME,
OBJECT_TYPE
)
Select
OBJECT_ID*-4 as OBJECT_ID,
PARENT_ID,
'Cloud Storage' as NAME,
905 as OBJECT_TYPE
From
OBJECTS_TREE
where OBJECTS_TREE.OBJECT_TYPE=322

UPDATE OBJECT_TYPES
set OBJECT_TYPE_GROUP=24000
WHERE OBJECT_TYPE= '905'

We have 243 guests and one member online