tFTPPut using FTPS Support for TLS/SSL Implicit encryption

[talendtester]

I am trying to make a Talend job to FTP a file to a server which uses TLS/SSL Implicit encryption.

I am able to login to the server and manually copy the file over successfully using WinSCP.

In my tFTPPut component what do I need to have checked or selected?

Do I need to have FTPS Support checked and Security Mode Implicit selected?

Do I need a Keystore File and a Keystore Password? What are these and how to I get or make them?

[Anonymous]

Hi,

Are your FTP Server using  FTPS (SSL/TLS)? The FTPS Support check box is used to connect to the FTP server via an FTPS connection. Once selected, you need to fill the Keystore File and Keystore Password fields.

Best regards

Sabrina

[vapukov · Accepted solution]

Yes for both:

- Yes You need check FTPS

- and yes You need provide path and password for You keystore (or just create new)

for create empty keystore (and truststore) You can check documentation:

https://docs.oracle.com/cd/E19509-01/820-3503/ggfen/

also You can use good tools like - http://keystore-explorer.org/features.html

[ron_sprenkels]

Hi,

I was in the exact same situation. In the end, I found this page: https://www.solypse.com/talend-ftps/ (in French).

I ended up doing these 5 steps:

1: generate an empty, local key store:

keytool -keystore clientkeystore -genkey -alias client -keystore local_keystore.jks

The keytool asks for a password to protect the keystore (twice). Give it one, at least 6 characters.

It also asks for firstname+lastname, organizational unit, organization, city, state, country. I went with the default 'Unknown' for all of them.

Then it asks for a password for the alias <client>, I went with the proposal to keep it the same as the password for the keystore.

You now have a keystore local_keystore.jks that has a private key for 'client'.

The tool issues a warning that we generated a keystore with a propietary format, and suggest a fix. Next step is to do just that fix.

2: convert the format of the freshly created keystore to type pkcs12:

keytool -importkeystore -srckeystorelocal_keystore.jks -destkeystorelocal_keystore.jks -deststoretype pkcs12

This step asks for the password of the keystore.

3: retrieved the certificate of the FTP server I want to connect to:

openssl s_client -connect 10.100.1.2:21 -starttls ftp </dev/null 2>/dev/null |openssl x509 -outform PEM > ftp_server.pem

4: convert that certificate from PEM into DEF format (just tested, could have done that directly in previous step)

openssl x509 -inform PEM -in ftp_server.pem -outform DEF -out ftp_server.cer

5: import the ftp_sever certificate into the keystore, indicating that it is thrusted:

keytool -import -file ftp_server.cer -keystore local_keystore.jks

The tool asks on the commandline if the certificate is to be trusted, answer 'yes'

You now have a keystorefile and password that can be used in a tFTPConnection Talend component, that will allow components like tFTPFileList and tFTPGet to access your FTP server. Be sure to check the 'FTPS Support (support tFTPGet temporarily)' checkbox, pick connection mode passive, and security mode implicit.

To find out all this took me several days, and a lot of frustration, so I hope I can prevent someone from having to go through the same, and thanks to the French guy for posting his original post.

Ron.

[Balz]

Hi Ron

Big thanks for your great tutorial! It saved a lot of work.

I simplified it a little bit down to 3 steps for a windows installation:

Requirements:

OpenSSL-Tool: For Windows get it for example here: https://slproweb.com/products/Win32OpenSSL.html

Solution:

1. Generate empty keystore type pkcs12
   CD "C:\Program Files\Java\jre1.8.0_211\bin" (in my case)
   keytool -keystore <Keystore-Name> -genkey -alias <Alias> -keystore <path+filename>.pks -deststoretype pkcs12 -keypass <Key-Password> -storepass <Store-Password>
   Leave the 5 questions blanc and answer the next one with J or Y (depends on language)
2. Download certificate type DEF
   CD "C:\Program Files\OpenSSL-Win64\bin" (in my case)
   openssl s_client -connect <FTP-Address>:21 -starttls ftp 2>NUL <NUL |openssl x509 -outform DEF > <path+filename>.cer
3. Fill the prepared keystore with certificate
   CD "C:\Program Files\Java\jre1.8.0_211\bin" (in my case)
   keytool -import -file <path+filename>.cer -keystore <path+filename>.cer.jks -keypass <Key-Password> -storepass <Store-Password>

Balz

[Anonymous]

Bonjour, 

we tried to apply such procedure but we still have issues : 

Starting job TestNewFTP at 12:36 24/06/2019.

[statistics] connecting to socket on port 3630

[statistics] connected

java.io.IOException: Invalid keystore format

      at sun.security.provider.JavaKeyStore.engineLoad(Unknown Source)

      at sun.security.provider.JavaKeyStore$JKS.engineLoad(Unknown Source)

      at java.security.KeyStore.load(Unknown Source)

      at flights.testnewftp_0_5.TestNewFTP$1MyTrust_tFTPConnection_1.getTrustManagers(TestNewFTP.java:4502)

      at flights.testnewftp_0_5.TestNewFTP$1MyTrust_tFTPConnection_1.access$0(TestNewFTP.java:4490)

      at flights.testnewftp_0_5.TestNewFTP.tFTPConnection_1Process(TestNewFTP.java:4522)

      at flights.testnewftp_0_5.TestNewFTP.tWarn_6Process(TestNewFTP.java:4400)

      at flights.testnewftp_0_5.TestNewFTP.runJobInTOS(TestNewFTP.java:5313)

      at flights.testnewftp_0_5.TestNewFTP.main(TestNewFTP.java:5054)

[2019-06-24 12:36]  Check context before upload : DEV

[2019-06-24 12:36]  Etape 2:

Exception in component tFTPPut_1

java.lang.NullPointerException

      at flights.testnewftp_0_5.TestNewFTP.tFTPPut_1Process(TestNewFTP.java:4775)

      at flights.testnewftp_0_5.TestNewFTP.tWarn_1Process(TestNewFTP.java:4688)

      at flights.testnewftp_0_5.TestNewFTP.tWarn_6Process(TestNewFTP.java:4418)

      at flights.testnewftp_0_5.TestNewFTP.runJobInTOS(TestNewFTP.java:5313)

      at flights.testnewftp_0_5.TestNewFTP.main(TestNewFTP.java:5054)

2019-06-24 12:36:31|orkFlJ|orkFlJ|orkFlJ|FLIGHTS|TestNewFTP|DEV|6|Java Exception|tFTPPut_1|java.lang.NullPointerException:null|1

[statistics] disconnected

Job TestNewFTP ended at 12:36 24/06/2019. [exit code=1]

any idea ?

[Anonymous]

---

---

I had the issue like Damien ( Invalid keystore format) and believe that this may have been caused by converting to PKCS12. I therefore created the keystore like this:

keytool -keystore c:\users\jason\desktop\clientkeystore -genkey -alias client

For the first/last name I used the same domain as my ftp server, but do not know if that is required. I was warned that JKS is proprietary.

I also had issues downloading the certificate with the other commands suggested (it would seem as if no certificate was there after a few minutes, creating a OKB file). I omitted the ftp and tls options (replace <hostname port> with your ftp server info):

openssl s_client -connect <hostname:port> 2>NUL <NUL |openssl x509 -outform DEF > c:\users\jason\desktop\newcert.cer

Then I was able to load the 'newcert.cer' into the 'clientkeystore' and use that in my tFTPConnection as the keystore (implicit) and use tFTPGet.

[phil974]

Hi,

I managed to configure tFTPConnect with your help.

But, i have an error message with tftpfile or tftget.

javax.net.ssl.SSLHandshakeException: Remote host closed connection during handshake

at sun.security.ssl.SSLSocketImpl.readRecord(Unknown Source)

at sun.security.ssl.SSLSocketImpl.performInitialHandshake(Unknown Source)

at sun.security.ssl.SSLSocketImpl.startHandshake(Unknown Source)

at sun.security.ssl.SSLSocketImpl.startHandshake(Unknown Source)

at org.apache.commons.net.ftp.FTPSClient._openDataConnection_(FTPSClient.java:646)

at org.apache.commons.net.ftp.FTPClient._openDataConnection_(FTPClient.java:785)

at org.apache.commons.net.ftp.FTPClient.initiateListParsing(FTPClient.java:3409)

at org.apache.commons.net.ftp.FTPClient.initiateListParsing(FTPClient.java:3339)

at org.apache.commons.net.ftp.FTPClient.listFiles(FTPClient.java:3016)

at sodiparc_bi.test_ftp_0_1.test_ftp$1FTPSGetter_tFTPGet_1.getFiles(test_ftp.java:590)

at sodiparc_bi.test_ftp_0_1.test_ftp.tFTPGet_1Process(test_ftp.java:782)

at sodiparc_bi.test_ftp_0_1.test_ftp.runJobInTOS(test_ftp.java:1079)

at sodiparc_bi.test_ftp_0_1.test_ftp.main(test_ftp.java:916)

Caused by: java.io.EOFException: SSL peer shut down incorrectly

Any idea ?

[Anonymous]

Hello,

Above Exception is a generic exception we get in the client if there is any of below scenario:

1. This issue probably occurs because of Internet connection problem/poor internet connection
2. You are missing your certificates.
3. If Server and Client support different version of TLS e.g. server support TLS2 while Client support only TLS1.

To know the exact cause of failure, you could enable

-Djavax.net.debug=all while executing your job?

Could you please also indicate on which talend build version you got this issue and what's JDK version are you using? Would you mind giving us some screenshots of your job setting? Which will be helpful for us to address your issue?

Please mask your sensitive data as well.

Best regards

Sabrina

[GuyWoollett]

Thanks for this work all those years ago..., used it today, worked a treat and saved me a bunch of time.