SFTP implementation question

Hi ETL-tools,

We are using Advanced ETL processor ENT 6.4.1.16.

We have a project using SFTP for file transfer with file content security requirements.

Would like to know how to implement in AETL?

Requirements are as below:
Each file (Data File) to be exchanged in the SFTP should be encrypted and accompanied with a Control File. The encrypted file is a binary file with “.enc” extension. The Control File is a fixed length text file describing the Data File and the encryption detail.
In addition to the Authentication Key-Pair, each SFTP Client works with below 2 key-pairs:  

No. Direction                                 Usage                                Algorithm / Key Length             Owner
1   File to be sent by us to Client  Encryption and Signing          RSA / 4096 bits                     Us
2   File to be sent by Client to us  Encryption and Signing          RSA / 4096 bits                    Client

Each Key-Pair is generated by the Owner. The Owner will send the Public Keys to peer via a manual procedure

Sender to Encrypt the File
For every file to be released to the SFTP server, the Sender performs the below steps:
1) Generate a random value with 32 bytes – this is the Nonce and act as a Symmetric Key.
2) Generate another random value with 16 bytes – this is the Initial Vector (IV).
3) Use the Nonce and IV to encrypt the Data File. The Encryption Algorithm is AES-256 (with CBC and PKCS5Padding options). The outcome of this step is the encrypted file (the filename with “.enc” extension, e.g. whatever_file.csv to whatever_file.csv.enc).
4) Encode the Nonce and IV to Base64 Strings and concatenate them with a “:” character.
5) Use the Receiver Public Key to Encrypt the concatenated string produced in 4) above. And convert the encrypted value into a Base64 String. This is the Encryption Parameter Set.
6) Store the Encryption Parameter Set produced in 5) above and also the date of using the Receiver Public Key to the Control File.
7) Calculate the SHA256 Hash Value of the Encrypted Data File (.enc file)
Use the Sender Private Key to encrypt the Hash produced in 7) above, and encode the encrypted Hash into a Base64 String. This is the Signature.
9) Store the Signature and the date of using the Sender Private Key to the Control File, and fill up other fields in the Control File.
10) The Encrypted Data File (whatever_file.csv.enc) and the Control File (whatever_file_check.txt) can be released to the SFTP Server.

Receiver to Decrypt the encrypted File
For every encrypted file received from the SFTP Server, the Receiver performs below steps:
1) Decode the Signature (which is a Base64 String) in the Control File, and use the Sender Public Key to decrypt (RSA, asymmetric) the decoded byte array. The outcome of this step is the Hash of the Encrypted Data File (.enc file). It should be noted that at the time of decrypting, a new Key-Pair may be effective. The Receiver should refer to the date stored in the Control File to determine the correct key to be used in RSA decryption.
2) Calculate the SHA256 Hash Value of the received Encrypted Data File (.enc file).
3) If the Hash derived from Step 1) above equal to the recalculated Hash in Step 2), the file integrity and sender are confirmed. Or otherwise, both Encrypted Data File and Control File are invalid and can be abandoned. This event should be logged for further reference.
4) Decode the Encryption Parameter Set (which is a Base64 String) in the Control File, and use the Receiver Private Key to decrypt (RSA, asymmetric) the decoded string (also beware of the date, see 1) above). The outcome of this step is the concatenated Base64 String of Nonce and IV and separated by the “:”.
5) Further break this concentrated Base64 String in 4) above by the separator, and decode the breakup strings. The Receiver then get the Nonce and IV.
6) Use the Nonce and IV to decrypt the Encrypted Data File (.enc file) by using AES-256 algorithm.
7) The original Data File is restored.
Further check the restore file length and the one stored in the Control File, if they are equal, the Data File is confirmed successfully restored. Or otherwise, both Encrypted Data File and Control File are invalid and can be abandoned. This event should be logged for further reference.

Thanks for your attention.

Kenny

Hi Kenny.

You have a very intersting challege and we would be delighed to help.

We will have a lok at at it and get back to you.

Most likely next week
 

Hi Mike,

Thanks for your reply.

BTW, please find the control file spec for reference.

Best regards,

Kenny

Hi Kenny.

1 Our software works with OpenPgp and It does support AES 256 Encryption

2  Can you clarify the below please

In addition to the Authentication Key-Pair, each SFTP Client works with below 2 key-pairs:  

No. Direction                                 Usage                                Algorithm / Key Length             Owner
1   File to be sent by us to Client  Encryption and Signing          RSA / 4096 bits                     Us
2   File to be sent by Client to us  Encryption and Signing          RSA / 4096 bits                    Client

I had a look at filezilla for example and I can see only one key file and not two 


 

Hi Mike,

The key should be refered to private key.

Please refer to the attached files for reference.

P.S. Trustee = Us, eMPF = Client

Best regards,

Kenny

Dear Team,

Want to know any update?

Best regards,

Kenny