Data Warehousing and Data integration

Extracting data form email body

More
2 years 11 months ago #22736 by AndrewRoberts
Hi,

An email comes in to an inbox a few times a day (example below). The body of the email has an introductory line, a couple of carriage returns, and then a section of JSON data I want to insert into a table. (I now can parse the data nicely thanks to Mike's separate feedback!).

I need to separate out the JSON data from the remainder of the email body.

Is the best way to do this to use Python? I can't seem to do it using the POP3 reader or package body elements. 

Kind regards, Andrew

Example email body:

Below are the registrants for our product.

[{"odata.etag":"W/\"datetime'2023-06-28T14%3A41%3A17.5015013Z'\"","PartitionKey":"6:2F28:2F2023","RowKey":"1609359378900:2E:5F47FED415:2D1E46:2D4F25:2DAFA6:2DFE573FB009FF","Timestamp":"2023-06-28T14:41:17.5015013Z","ProductId":"1609359378900.xxxx","CustomerInfo":"{\"FirstName\":\"xxxx\",\"xxxx\":\"xxxx\",\"Email\":\"xxxx\",\"Phone\":\"xxxx\",\"Country\":\"xxxx\",\"Company\":\"xxxx\",\"Title\":null}","LeadSource":"xxxx","ActionCode":"INS","PublisherDisplayName":"xxxx","OfferDisplayName":"xxxx","CreatedTime":"06/28/2023 14:41:17","Description":""},{"odata.etag":"W/\"datetime'2023-06-28T14%3A01%3A43.0817537Z'\"","PartitionKey":"6:2F28:2F2023","RowKey":"1609359378900:2E:5F8205ECB0:2D5E7D:2D4E0F:2D8E86:2DD80B211A24B1","Timestamp":"2023-06-28T14:01:43.0817537Z","ProductId":"1609359378900.xxxx","CustomerInfo":"{\"FirstName\":\"yyyy\",\"LastName\":\"yyyy\",\"Email\":\"yyyy\",\"Phone\":\"yyyy\",\"Country\":\"yy\",\"Company\":\"yyyy\",\"Title\":null}","LeadSource":"yyy","ActionCode":"INS","PublisherDisplayName":"yyyy","OfferDisplayName":"yyyy","CreatedTime":"06/28/2023 14:01:42","Description":""}]
 

Please Log in or Create an account to join the conversation.

More
2 years 11 months ago #22737 by Peter.Jonson
Can we ask you to forward us a couple of emails?

Working with an email body can be tricky because of how it is encoded.

IMAP4 reader may work but it is not 100% guaranteed

Peter Jonson
ETL Developer

Please Log in or Create an account to join the conversation.

More
2 years 11 months ago #22738 by AndrewRoberts
Sure - to which email address Peter?
The following user(s) said Thank You: Peter.Jonson

Please Log in or Create an account to join the conversation.

More
2 years 11 months ago #22739 by Peter.Jonson
support@etl-tools.com

Peter Jonson
ETL Developer

Please Log in or Create an account to join the conversation.

More
2 years 11 months ago - 2 years 11 months ago #22740 by Peter.Jonson
Hi Andrew

I have created a transformation which extracts JSON data from emails message

Please see the screenshots below.

You would need to be careful with it because if the supplier changes the format it will stop working 

 

 

 

Peter Jonson
ETL Developer
Last edit: 2 years 11 months ago by Peter.Jonson.

Please Log in or Create an account to join the conversation.

More
2 years 11 months ago #22741 by Peter.Jonson
The following user(s) said Thank You: AndrewRoberts

Please Log in or Create an account to join the conversation.

Cookies user preferences
We use cookies to ensure you to get the best experience on our website. If you decline the use of cookies, this website may not function as expected.
Accept all
Decline all
Read more
Marketing
Set of techniques which have for object the commercial strategy and in particular the market study.
Google
Accept
Decline
Analytics
Tools used to analyze the data to measure the effectiveness of a website and to understand how it works.
Google Analytics
Accept
Decline
Google Analytics
Accept
Decline
Functional
Tools used to give you more features when navigating on the website, this can include social sharing.
Advertisement
If you accept, the ads on the page will be adapted to your preferences.
Google Ad
Accept
Decline
Save