Databricks projects

[t_schueller]

Hello,

I created two C4DL Projects by choosing Databricks as type.

First project: I was surprised when I took a look at my DB and saw the data format, so I checked the project type and it's setted to Historical Data Store. You can see it the following screenshot and the deployment package extract:

"exclude_to_date": true,
"acid_transactions": true,
"big_data_project_type": "HISTORICAL_DATASTORE_WITHOUT_ACID",
"dwh_type": "DATABRICKSDELTA",
"rno_ins": "RUNNO_INSERT",
"rno_upd": "RUNNO_UPDATE",

The second project: The project type is setted to Operational Data Store.

"exclude_to_date": true,
"acid_transactions": true,
"big_data_project_type": "OPERATIONAL_DATASTORE",
"dwh_type": "DATABRICKSDELTA",
"rno_ins": "RUNNO_INSERT",
"rno_upd": "RUNNO_UPDATE",

  

The documentation says:

Databricks is currently supported on Azure. Delta is required by Compose in order to
create a replica of the source data (Operational Data Store - ODS).

I created both project the same way and I can't select an other options than ODS.

Can you explain me what happend?

Thanks for your help.

[Brian_Jones]

I can't fully explain how one of your Compose for Data Lakes projects had a project type of HISTORICAL_DATASTORE_WITHOUT_ACID in the json file , but I can confirm the only type of Compose for Data Lakes Databricks project you should be able to create is an ODS. Perhaps you can create a ticket and upload the two project json files so our support team can investigate further?

[t_schueller]

Hi Brian,

Thanks, I will open a case in the support