Databricks Lakehouse (Delta) Target connection not working in DRP server.

[SaidSayyad]

Hello Team,

we are using Databricks Lakehouse (Delta) as a target. while setup connection we are observing below error.

we tried to uninstall & re-install  Simba Sparks driver version simbaspark-2.6.26.1045-1.x86_64.rpm. we also updated Simba file but still facing same error.

Update file /opt/simba/spark/Setup/odbcinst.ini

Add lines :

# Driver from the SimbaSparkODBC-2.6.29.1049-LinuxRPM-64bit package

# Setup from the unixODBC package

[Simba Spark ODBC Driver]

Description     = Simba Apache Spark ODBC Connector (64-bit)

Driver          = /opt/simba/spark/lib/64/libsparkodbc_sb64.so

Test connection Error: 

• Failed prepare Cloud component
• Cannot connect to Cloud server RetCode: SQL_ERROR SqlState: 01000 NativeError: 0 Message: [unixODBC][Driver Manager]Can't open lib 'Simba Spark ODBC Driver' : file not found

Logfile error:

01958621: 2023-09-29T10:32:27:706457 [STREAM_COMPONEN ]I: Going to connect to server westeurope.azuredatabricks.net database (null) (cloud_imp.c:3883)

01958621: 2023-09-29T10:32:27:706497 [STREAM_COMPONEN ]I: Target endpoint 'Databricks Lakehouse (Delta)' is using provider syntax 'DatabricksDelta' (provider_syntax_manager.c:894)

01958621: 2023-09-29T10:32:27:719774 [STREAM_COMPONEN ]E: RetCode: SQL_ERROR SqlState: 01000 NativeError: 0 Message: [unixODBC][Driver Manager]Can't open lib 'Simba Spark ODBC Driver' : file not found [1022502] (ar_odbc_conn.c:584)

01958621: 2023-09-29T10:32:27:719788 [STREAM_COMPONEN ]E: Cannot connect to Cloud server [1022501] (cloud_imp.c:3918)

01958621: 2023-09-29T10:32:27:719790 [STREAM_COMPONEN ]E: Failed prepare Cloud component [1022501] (cloud_metadata.c:239)

Thanks,

Said Sayyad

[john_wang]

Hello @SaidSayyad ,

Thanks for reaching out to Qlik Community!

Looks to me this is an ODBC configuration issue. Let's troubleshoot the issue by steps:

1. Please merge below lines into "/etc/odbcinst.ini" file rather than using "/opt/simba/spark/Setup/odbcinst.ini"

[Simba Spark ODBC Driver]
Description = Simba Apache Spark ODBC Connector (64-bit)
Driver = /opt/simba/spark/lib/64/libsparkodbc_sb64.so

2. Add the PATH "/opt/simba/spark/lib/64/" to LD_LIBRARY_PATH environment variable in file "/opt/Attunity/replicate/bin/site_arep_login.sh" (default location). Make sure the Replicate account (by default the name is "attunity") has enough privileges to access/execute the Simba files.

3. Restart Qlik Replicate Service after the above step 2 is done.

4. If the problem persists, it's better to create 64-bit ODBC DSN in file "/etc/odbc.ini" and then use command "isql" to test the connectivity issue, it's much easier than doing it by Replicate. If "isql" works fine then Replicate should work as well.

5. Link from Databricks: Troubleshooting JDBC and ODBC connections

Hope this helps.

Regards,

John.

[SaidSayyad]

Hello John,
I have performed all steps provide by you but still facing same issue. Please find attached snapshot for your reference.

Thanks,
Said Sayyad

[john_wang · Accepted solution]

Hi @SaidSayyad ,

Thanks for the information.

The default driver name is "Simba Spark ODBC Driver". In your "odbcinst.ini" file, the driver name is "Simba Spark ODBC Driver 64-bit". So far you can either rename the driver name in "odbcinst.ini" file to default name, or you can use the predefined name in Databricks target endpoint, as below:

1. Open Databricks target endpoint
2. Go to the Advanced tab
3. Go to the Internal properties
4. Add parameter name "driver" in the input label, and set its value to "Simba Spark ODBC Driver 64-bit".
   
   

If the problem persists, please try to use "isql" to test the connectivity issue.

BTW, it seems the Samba folder was not added into LD_LIBRARY_PATH environment variable correctly: the line is commented out. please use "env|grep  LD_LIBRARY_PATH " to check if it's set correctly.

Regards,

John.

[SaidSayyad]

Thank you John,

Now connection is working fine, you can mark this case as close.



Thanks,

Said Sayyad

[SushilKumar · Accepted solution]

Hello team,

If our response has been helpful, please consider clicking "Accept as Solution". This will assist other users in easily finding the answer.

Regards,

Sushil Kumar

[john_wang]

Hello @SaidSayyad ,

Thank you so much for your feedback.

Do you mind sharing with us the final solution? It's very helpful for all of us.

Best Regards,

John.

[SaidSayyad · Accepted solution]

Hello John,

As per your suggestion we have changed driver name under  "odbcinst.ini" file and it's working now.

Thanks,

Said Sayyad

[john_wang]

Hi @SaidSayyad ,

Glad to hear that and thanks for your update.

Regards,

John.