Qlik Replicate: How to run curl command to connect hdfs using Active directory Kerberos authentication and SSL?

[SwathiPulagam]

If you have any connectivity issues between the Qlik Replicate endpoint and the Hadoop server then follow the below troubleshooting steps outside of replicate but the commands must be run on the same server where replicate is installed.

Explained in this article is one of the most complex setups which is Active Directory Kerberos Authentication + SSL to connect hdfs.

1. Run Active Directory Kerberos Authentication Request as below:
   
   

   C:\Windows\system32>"C:\Program Files\Attunity\Replicate\jvm\bin\kinit" <PRINCIPLE NAME>@<RELAM NAME>

   Password for <PRINCIPLE NAME>@<RELAM NAME>:

   New ticket is stored in cache file C:\Users\<USER NAME>\krb5cc_<USER NAME>

2. Set KRB5CCNAME variable with the newly generated Kerberos ticket.
   
   

   Set KRB5CCNAME for the session as below:

   SET KRB5CCNAME=C:\Users\<USER NAME>\krb5cc_<USER NAME>

3.  Set Environment variable PATH for curl.exe location so we can run Curl command from any place as below:
   
   curl -v -i -k -L --negotiate -u : --cacert "C:\Program Files\Attunity\Replicate\ssl\certs\PEM\hdfsssl.pem" -H 'Content-Type: application/json;charset=utf-8' -X PUT -d @XXXXXXXXX.csv https://<hdfs node name>:<port>/webhdfs/v1/apps/hive/warehouse/<dbname>/<table name>/XXXXXXXXX.csv?op=CREATE&data=true&buffersize=67108864&overwrite=true

Once all the above steps work outside of replicate and you still encountering connectivity issues then contact support through the support portal.

Environment

Qlik Replicate