topic Re: Databricks projects in Qlik Compose

Databricks projects

t_schueller — Thu, 20 Mar 2025 15:56:15 GMT

Hello,

I created two C4DL Projects by choosing Databricks as type.

First project: I was surprised when I took a look at my DB and saw the data format, so I checked the project type and it's setted to Historical Data Store. You can see it the following screenshot and the deployment package extract:

"exclude_to_date": true,
"acid_transactions": true,
"big_data_project_type": "HISTORICAL_DATASTORE_WITHOUT_ACID",
"dwh_type": "DATABRICKSDELTA",
"rno_ins": "RUNNO_INSERT",
"rno_upd": "RUNNO_UPDATE",

The second project: The project type is setted to Operational Data Store.

"exclude_to_date": true,
"acid_transactions": true,
"big_data_project_type": "OPERATIONAL_DATASTORE",
"dwh_type": "DATABRICKSDELTA",
"rno_ins": "RUNNO_INSERT",
"rno_upd": "RUNNO_UPDATE",

The documentation says:

Databricks is currently supported on Azure. Delta is required by Compose in order to
create a replica of the source data (Operational Data Store - ODS).

I created both project the same way and I can't select an other options than ODS.

Can you explain me what happend?

Thanks for your help.

Re: Databricks projects

Brian_Jones — Wed, 09 Dec 2020 22:20:03 GMT

I can't fully explain how one of your Compose for Data Lakes projects had a project type of HISTORICAL_DATASTORE_WITHOUT_ACID in the json file , but I can confirm the only type of Compose for Data Lakes Databricks project you should be able to create is an ODS. Perhaps you can create a ticket and upload the two project json files so our support team can investigate further?

Re: Databricks projects

t_schueller — Wed, 09 Dec 2020 22:26:03 GMT

Hi Brian,

Thanks, I will open a case in the support