Pre-Requisites for Migrating Projects from Compose for Data Lakes to Qlik Compose

[Troy_Raney]

This is part 1 of 4 in a series of articles with information about migrating projects from Compose for Data Lakes 6.6 (C4DL 6.6) to Qlik Compose 2021.8 (Gen2)

Different Migration Scenarios for customers

• Databricks to Databricks
  • Migrate Databricks projects from Compose for Data Lakes 6.6 to Qlik Compose 2021.8
• Hive to Hive supported version
  • Migrate Hive projects from Compose for Data Lakes 6.6 to Qlik Compose 2021.8
• Spark to Spark supported version
  • Migrate Spark projects from Compose for Data Lakes 6.6 to Qlik Compose 2021.8

NOTE:

1) No more remote agent software exists for Qlik Compose

2) you can use same license key from C4DL, contact support if you have any trouble.

Please go to below link and make sure your environment has supported database versions for Data Lake projects.

https://help.qlik.com/en-US/compose/August2021/Content/ComposeDWDL/Main/DW/Supported%20Platforms%20a...

Module 1: Pre-requisites for Migration and How to handle ODS projects

• Install Qlik Compose on a dedicated machine\the same machine where C4DL 6.6 exist.
• Make sure that speed partition option enabled on replicate task.
• High Level Migration flow as below.
  • Create deployment package in C4DL 6.6 for each project.
  • create project in Gen2:
    • Create a new DL project with the new project wizard in Qlik Compose 2021.8
    • define Storage (if you decided to migrate data also then you should use new database name)
  • Run Qlik Compose cli to provide c4dl deployment package as an input and that will create new adjusted deployment zip file into deployment_package folder.

Ex: ComposeCli.exe adjust_cfdl_project --project Hive_ODS --infile “C:\Program Files\Qlik\C4DL66\ Hive_ODS_deployment_<datetime>.zip”

 à C:\Program Files\Qlik\Compose\data\projects\Hive_ODS\deployment_package\Hive_ODS __<datetime>__QlikComposeDLMigration.zip

• import adjusted deployment package to the new project we just created in Qlik Compose.
• verify that you can work with this project- create the storage, generate tasks.
• Finally, Migrate the data using below steps.
  • Generate 'create data migration script’ using below cli command and run it on gen2 storage db.

Ex: ComposeCli.exe create_cfdl_data_migration_script --project Hive_ODS --infile       “C:\Program Files\Qlik\C4DL66\Hive_ODS_deployment_<datetime>”

• verify the data is correct

NOTE:

1)Storage database names must be different for c4dl 6.6 and gen2 because we are going to run select * from C4DL 6.6 storage database and insert into Qlik Compose 2021.8 storage database in the migrate data step.

2) If you use above process, you can run both c4dl 6.6 and gen2 projects parallelly for few days until your downstream apps comfortable with using gen2 views.

• In case of ODS project, we suggest you go with compose full load and CDC once the replicate task reload finished. You can only migrate project deployment and skip data migration here. This can be applicable for HDS projects also if you are ok with replicate reload and loosing historical data form storage DB. Below is the command to migrate project definition for Hive ODS.

Ex: ComposeCli.exe adjust_cfdl_project --project Hive_ODS --infile “..\Qlik\C4DL66\Hive_ODS_deployment_xx.zip”

You can watch demo video How to handle ODS projects or for any project if you decide to reload replicate task.

Environment

• From Compose for Data Lakes 6.6
• To Qlik Compose 2021.8

The information in this article is provided as-is and to be used at own discretion. Depending on tool(s) used, customization(s), and/or other factors ongoing support on the solution below may not be provided by Qlik Support.

Related Content 

Migration paths for Qlik Compose