Slemma is a simple business intelligence software that helps modern businesses leverage data into advantages online with less effort. Plug in directly to the most popular databases and cloud services; create, customize, and share dashboards and visualizations with the entire team.
All our ETL products support now loading data into slemma
Based on the customer feedback in the latest version of Advanced ETL Processor we have introduced support for JSON
JSON is much more efficient than XML. It reqires less network traffic processing power and memory to work with
"Drinks": ["Water", "Beer", "Vine"]
"complaint": "Blured night vision"
Quite often people ask us how to stop all ETL packages execution.
Most of the time, this is because Professional version is being used to execute the packages.
Better option is to use Enterprise version agent and run tasks in parallel.
To stop execution
This work for both Advanced ETL Processor and Visual Importer ETL
An external PLC generates txt files and those must be downloaded and converted into an SQL DB.
The PLC generates 4 files each day, at a size of 5MB pr. File = 20MB a day.
The file name is date and time named like this : 14022719.txt, syntax is YYMMDDHH
The files are generated at the same 4 hours each day, 01, 07, 13, 19
In the files, first data row is a time stamp YYMMDDHHMMSS
It is important that one record with a specific date/time is written only once in the SQL.
The files would always be the same size.
Each day, The ETL converter should copy the file to a local directory via FTP; convert the file to SQL, then delete the file from the local dir. It could be 4 times a day, maybe one hour after the txt file is completed on the PLC.
We can not delete the file from the PLC, since others should also have access to the data.
The data on the PLC will be deleted after 10 days using FIFO
How to I keep track of which files I have copied ?
Could create a TXT file on local drive and write the file names, compare with whats on the FTP drive ?
But how do we do this ? That is file comparison..can ETL do this ?
Is there any other way to do this ?
Keep in mind, it could be that the internet connection is off line for a day or 2, the server could be down,
The PLC could be down. In some situations you will need to copy 4 files, some times 6 files..depents..
Or you could have gaps in the time stamps of the file names from the PLC if it has been down for a while.
I know we would then also have gaps in the SQL, but that is OK.