Qlik Compose for Data Warehouses

[mopidesp]

Can you please clarify below questions ?

1. We have salesforce as one of the sources. Some tables from salesforce has more than 300 columns. When we discover the table in to compose we could see only 221 columns. Below are few examples.
   source table 191 columns - Qlik Compose picks only 178
   source table 78 columns - Qlik Compose  picks only 69
   Is there any limitation in Compose regarding the number of columns ?

1. We have Data vault and DataMart built and loaded via compose. If I want to truncate and reload all of the data mart tables with out doing the full load build (as full rebuild is dropping the tables), Can I delete TPIL_DMA_RUNNO (  where DWH_RUNNO ! ='-1') table and run the data mart process ? If there are any other options for this, please advise.

[pandabrowski]

hi, is there a chance, that there are fields that are not supported by compose?

usually you can convert it during replication for the proper data types.

please check the data types of missing columns

best regards, rd

[TimGarrod]

As suggested above Compose does not support all data types.   I would discover one of the tables and in the screen that is shown during discovery, review the messages to see if any columns have been omitted due to data types. 

1. We have Data vault and DataMart built and loaded via compose. If I want to truncate and reload all of the data mart tables with out doing the full load build (as full rebuild is dropping the tables), Can I delete TPIL_DMA_RUNNO (  where DWH_RUNNO ! ='-1') table and run the data mart process ? If there are any other options for this, please advise.

Is this a one-time thing or ongoing?  

If you wish the data mart to always be a rebuild - there is a setting in the DataMart - for FULL REBUILD or Incremental.   You can configure this, Generate code and run - Compose will reload the tables. 

This is a better practice than reseting the TPIL table.  If you simply DELETE - Compose will mark records obsolete in the Dim (but leave them there) and insert the Dim records and you'll have to manage "OBSOLETE" records.  (Please review the documentation on OBSOLETE INDICATOR at help.qlik.com). 

You should never DELETE from TPIL_DMA_RUNNO. 

If you do wish to have a 1 time "refresh" , my recommendation would be to truncate the DIM and FACT tables and set the DWH_RUNNO for that PSET to -1.  This is the inital state of the TPIL_DMA_RUNNO table.  If you delete then Compose does not re-insert the data into that table and EVERY run will be a complete refresh.

update EDW.dbo.TPIL_DMA_RUNNO
set DWH_RUNNO = -1
WHERE PSET_NUMBER =   (THE PSET_NUMBER FOR THE DM YOU ARE PROCESSING)

[mopidesp]

Thanks Tim.

Truncating and reloading data mart is a one time thing and not on daily basis. I will try the option you have suggested and will let you know if have any questions. 

Thanks again for your time.

[TimGarrod]

No problem. Also note - you can in the GUI drop and recreate the data mart only. Generate and run.

If in dev that’s an easy way to rebuild 🙂