Execution stuck

We use Advanced ETL Processor to download emails. We process thousands of emails per day.

From time to time it gets stuck for no reason and It runs past timeout. 

It seems to happen on and off, only if I log in and run the below about 5 times to get it back on track

Taskkill /IM aetlcl.exe /F

Any suggestions   

Yes looks like it is getting stuck for some reason.It should be possible to kill from the app itself.We will have a look and let you know

Can you try the latest release please.

Here is a full list of changes

07/10/2024 6.4.4.26

 + Improved: Execution Agent Timeout Handling 
 + Added: Encript Dynamic Package Action
 + Added: Decript Dynamic Package Action
 - Fixed: SFTP List Sorting

Oscar wrote:

We use Advanced ETL Processor to download emails. We process thousands of emails per day.
 

HI Oscar, 
This is an awesome use case , care to elaborate on this ? I would love to see or understand your workflow as I have never explored AETL for processing emails.

Hi Prashant

We developed an email processing workflow for one of our customers, designed to streamline the handling of financial statements. While full automation wasn't achievable due to various challenges, this solution significantly reduces manual effort for the customer.

The workflow processes financial statements, which are received either as Excel files or PDFs. When PDFs are received, they are converted to Excel format.The first step is linking the email to the corresponding customer account. This is done by checking the email domain of the sender and analyzing the email body, especially for customers who forward emails. In cases where an email has been forwarded multiple times, it can be difficult to determine if the statement belongs to Customer 1 or Customer 2. To address this, the system provides an input dialogue that allows users to select the correct email domain.

Each statement comes in a unique format.

For example:

• For Customer 1, "Invoice Date" is found in column A, and "Amount" in column B.
• For Customer 2, "Invoice Date" is in column B, and "Amount" is in column K.

We also rename the sheets as part of the process. The mapping information for each customer is stored in a lookup table, which is then used by the transformation process to convert the data into a standard format before loading it into the database. 

OK WOW !!

1. Thanks for posting this ,I honestly didn't know I could make big packages in ETL , this is very useful to people like me , who are always concerned with , is my package too big now
2. This is the kind of package which should have it's own YouTube video to show the real POWER usage of this 
3. In Covid , we had a massive decrease in manpower available to us , we had to do something similar but not this level , we

• Used TheBat! from Ritlabs to save attachment to specific directory which matched the bank websites 
• They were PDF too , once they were received , they would kickstart a macro in Keyboard Maestro
• Firstly each file was converted from PDF to text or CSV using PDF2text library 
• Each CSV was matched painfully with a series of 15 regular expression rules
• Once matched , all of them were extracted form Advise,Invoice, Export Payment, Import Payment
• The result was then stored in a CSV for easy view
• File was uploaded to Gdrive

3.1 I think we spent 45 days to develop this , if I had AETL I think it would have taken 15-20days  and even more robust