Need suggestions in Data-model

[paulwalker]

Hi Experts,

I'm facing issue with Data model,  I have 10 transaction tables tables, users need all the data from tables.

Which data model should i approach ?

Current instance, have done concatenate all the tables using with flag and creating QVD. but taking more than hour.

I feel , Few tables has same fields and few tables are not common fields, so taking more time to create QVD.

Please can you give suggestions.

Thanks in Advance.

Regards,

Paul

[PrashantSangle]

How many records do you have in each transaction table???

Regards,
Prashant Sangle

[tresesco]

In such scenarios, usually one goes with concatenation or link-table solution. The way you are currently going (concatenation) gives you better performance at the UI level, because you can use your flags in set analysis expressions. However, since there is no general rule (concatenation or link-table being one better than other), you could try alternative  link-table solution and see if that helps in your use case. for better understanding ,please refer: Concatenate or Link Table

[marcus_sommer]

Is it really a question in regard to your datamodel (I think your concatenate-approach is a good choice) or more how to speed up the load-process? Quite useful for reducing the loading-times is to use incremental load-approaches and this mustn't be restricted to the rawdata-level else it might be necessary to implement it for each transformation-level. Within the second half of the following posting you will find various links to incremental-examples and exists() which is needed to keep the qvd-loadings optimized: Advanced-topics-for-creating-a-qlik-datamodel.

- Marcus

[paulwalker]

I did incremental load in QVD Generator and working fine.

These concatenate using all QVD's only, Loading process taking hardly 11 mins, but QVD creating more than 40 mins.

[marcus_sommer]

Are all QVD's loading optimized within the incremental generator-process and by concatenate them? Are all transformations included within the incremental approach?

- Marcus

[paulwalker]

QVD generator is complete RAW data, no transformations. (fetching RAW data from SQL)

ETL: I did few transformations and concatenated all the QVD's in to one (Finally creating one QVD).

-- Paul

[marcus_sommer]

It does not really answer the above questions if all (respectively the heavy ones) loadings are optimized and all transformations are included within an incremental load-approach.

Beside this your answer sounds that there is one generator- and one datamodel-application. If it's the case it might be useful to split them into multiple applications because they could then run in parallel (depending on the performance of your database/network and your resources of CPU, RAM and storage and/or the number of available machines). Further not each single-task might need to be updated each time and might be therefore skipped n times.

- Marcus

[paulwalker]

Hi Experts,

I have attached sample data for 2 tables, please anyone can help on Link table.

-Paul