Unlock a world of possibilities! Login now and discover the exclusive benefits awaiting you.
Running a Talend Job using a key pair authentication for Snowflake fails with the exception:
Starting job Snowflake_CreateTable at 09:21 19/07/2021. [statistics] connecting to socket on port 3725 [statistics] connected Exception in component tDBConnection_2 (Snowflake_CreateTable) java.lang.RuntimeException: java.io.IOException: Missing Keystore location at edw_demo.snowflake_createtable_0_1.Snowflake_CreateTable.tDBConnection_2Process(Snowflake_CreateTable.java:619) at edw_demo.snowflake_createtable_0_1.Snowflake_CreateTable.runJobInTOS(Snowflake_CreateTable.java:3881) at edw_demo.snowflake_createtable_0_1.Snowflake_CreateTable.main(Snowflake_CreateTable.java:3651) [FATAL] 09:21:38 edw_demo.snowflake_createtable_0_1.Snowflake_CreateTable- tDBConnection_2 java.io.IOException: Missing Keystore location java.lang.RuntimeException: java.io.IOException: Missing Keystore location at edw_demo.snowflake_createtable_0_1.Snowflake_CreateTable.tDBConnection_2Process(Snowflake_CreateTable.java:619) [classes/:?] at edw_demo.snowflake_createtable_0_1.Snowflake_CreateTable.runJobInTOS(Snowflake_CreateTable.java:3881) [classes/:?] at edw_demo.snowflake_createtable_0_1.Snowflake_CreateTable.main(Snowflake_CreateTable.java:3651) [classes/:?]
The Keystore path is not configured correctly at the Job or Studio level before connecting to Snowflake DB on the metadata and using the same metadata connection in the Jobs.
To use key pair authentication for Snowflake, the Keystore settings must be configured in Talend Studio before connecting to Snowflake.
Perform one of the following options.
Update the appropriate Studio initialization file (Talend-Studio-win-x86_64.ini,Talend-Studio-linux-gtk-x86_64.ini,or Talend-Studio-macosx-cocoa.ini depending on your operating system), with the following settings:
-Djavax.net.ssl.keyStore={yourPathToKeyStore} -Djavax.net.ssl.keyStoreType={PKCS12}/{JKS} -Djavax.net.ssl.keyStorePassword={keyStorePassword}
Update the Keystore configuration in Studio SSL preferences with the required Path, Password, and Keystore Type.
Add the Key Alias to the Snowflake metadata.
Key Alias will be created with your keystore .p12 file
Update the tSetKeystore components in your Job, if you plan to run the Job when the target execution is local, Remote Engine, or JobServer (the versions do not matter). Before selecting the Key Pair option for the tSnowflakeConnection component, configure the key pair authentication on the Basic settings tab of the tSetKeystore component:
Select JKS from the TrustStore type pull-down list.
Enter " " in the TrustStore file field.
Clear the TrustStore password field.
Select the Need Client authentication check box.
Enter the path to the Keystore file in double quotation marks in the KeyStore file field.
Enter the Keystore password in the KeyStore password field.
If you are looking for how to set up key-pair authentication in Snowflake using Qlik Talend Studio step by step, please refer to this article:
Qlik-Talend-Product-How-to-set-up-Key-Pair-Authentication-in
Hello @Xiaodi_Shi
Thanks for the reply, I'm expecting your help there. Thanks
Regards
Fazil M
I am trying to set-up snowflake key-pair authentication from Talend open studio
Talend Version: 8.0.1
Steps followed:
1) generated private and public keys using openssl ( windows)
2) Altered snowflake user with public key
3) stored rsa_key.p8 in talend machine
4)
Based on the thread discussion, I came to know that .p8 is not supported from talend and snowflake does support only .pem (.p8).
Do I need to convert .p8 to .jks and point .jks in keystore component and keeping .p8 on snowflake end
Talend .jks format--------> snowflake .p8
do we need passphrase for private key when connecting from talend ?
please check above understanding is correct
Hello @venkattalend
If your PSCS12 version keystore ended with .pem, please convert this private key to PKCS12 format and use KeyStore type PKCS12 in tSetKeystore component. In Snowflake, assign the public key to the Snowflake user and run your job in studio.
Feel free to let us know if your connection go through and data could be read from the DB.
Best regards
Sabrina
@Xiaodi_Shi Understood. What you're saying is that first, I generate keys using OpenSSL in .pem format, and then convert the .p8 to PKCS#12.
let me try and update the group. thanks
@Xiaodi_Shi I tried below steps, and finally got error
"New public key rejected by current policy. Reason: 'Invalid Public key' "
can you provide detailed commands to configure the setup
step1
--Generate .jks keystore
keytool -genkey -keyalg RSA -v -keystore keystore.jks -alias first-key-pair
step2
---jks keystore to PKCS12
keytool -importkeystore -srckeystore C:\Users\etldeveloper\Desktop\keypair\keystore.jks -srcstoretype JKS -destkeystore C:\Users\etldeveloper\Desktop\keypair\keystore.p12 -deststoretype PKCS12
step3
--PKCS12 to pem
openssl pkcs12 -in C:\Users\etldeveloper\Desktop\keypair\keystore.p12 -out C:\Users\etldeveloper\Desktop\keypair\keystore.pem
step4
openssl x509 -in certificate.crt -out public_key.pem -outform PEM
Hello @venkattalend
Here is a good example with commands step by step for converting a PEM File to Java KeyStore Format
https://www.baeldung.com/convert-pem-to-jks
Please feel free to let me know if it helps.
Best regards
Sabrina
Hi @Xiaodi_Shi
I was able to connect successfully using link https://www.baeldung.com/convert-pem-to-jks
I was able to query data by putting query in "Manual query" but when i try to load tables in sowflake input component it throwing error, please check attached screenshot
what could be the reason?
@venkattalend could you please share a screenshot of the basic settings of tSnowFlakeConnection? You mentioned that you were able to query a small data set using Manual Query, but an error occurred when loading a large data set, right?