Qlik Community

Ask a Question

Qlik Compose for Data Lakes Discussions

Discussion board for collaboration on Qlik Compose for Data Lakes.

Announcements
QlikWorld Online 2021, May 10-12: Our Free, Virtual, Global Event REGISTER TODAY
cancel
Showing results for 
Search instead for 
Did you mean: 
t_schueller
Partner
Partner

Databricks projects

Hello,

I created two C4DL Projects by choosing Databricks as type.

First project: I was surprised when I took a look at my DB and saw the data format, so I checked the project type and it's setted to Historical Data Store. You can see it the following screenshot and the deployment package extract:

"exclude_to_date": true,
"acid_transactions": true,
"big_data_project_type": "HISTORICAL_DATASTORE_WITHOUT_ACID",
"dwh_type": "DATABRICKSDELTA",
"rno_ins": "RUNNO_INSERT",
"rno_upd": "RUNNO_UPDATE",

The second project: The project type is setted to Operational Data Store.

"exclude_to_date": true,
"acid_transactions": true,
"big_data_project_type": "OPERATIONAL_DATASTORE",
"dwh_type": "DATABRICKSDELTA",
"rno_ins": "RUNNO_INSERT",
"rno_upd": "RUNNO_UPDATE",

Project1.png

Project2.png

  

The documentation says:

Databricks is currently supported on Azure. Delta is required by Compose in order to
create a replica of the source data (Operational Data Store - ODS).

I created both project the same way and I can't select an other options than ODS.

Can you explain me what happend?

 

Thanks for your help.

Labels (1)
2 Replies
Brian_Jones
Employee
Employee

I can't fully explain how one of your Compose for Data Lakes projects had a project type of HISTORICAL_DATASTORE_WITHOUT_ACID in the json file , but I can confirm the only type of Compose for Data Lakes Databricks project you should be able to create is an ODS. Perhaps you can create a ticket and upload the two project json files so our support team can investigate further?

t_schueller
Partner
Partner
Author

Hi Brian,

Thanks, I will open a case in the support