topic Re: Need suggestions in Data-model in QlikView

Need suggestions in Data-model

paulwalker — Wed, 25 Nov 2020 16:16:04 GMT

Hi Experts,

I'm facing issue with Data model, I have 10 transaction tables tables, users need all the data from tables.

Which data model should i approach ?

Current instance, have done concatenate all the tables using with flag and creating QVD. but taking more than hour.

I feel , Few tables has same fields and few tables are not common fields, so taking more time to create QVD.

Please can you give suggestions.

Thanks in Advance.

Regards,

Paul

Re: Need suggestions in Data-model

PrashantSangle — Wed, 13 Feb 2019 06:49:28 GMT

How many records do you have in each transaction table???

Regards,
Prashant Sangle

Re: Need suggestions in Data-model

tresB — Wed, 13 Feb 2019 06:54:47 GMT

In such scenarios, usually one goes with concatenation or link-table solution. The way you are currently going (concatenation) gives you better performance at the UI level, because you can use your flags in set analysis expressions. However, since there is no general rule (concatenation or link-table being one better than other), you could try alternative link-table solution and see if that helps in your use case. for better understanding ,please refer: Concatenate or Link Table

Re: Need suggestions in Data-model

marcus_sommer — Wed, 13 Feb 2019 08:23:28 GMT

Is it really a question in regard to your datamodel (I think your concatenate-approach is a good choice) or more how to speed up the load-process? Quite useful for reducing the loading-times is to use incremental load-approaches and this mustn't be restricted to the rawdata-level else it might be necessary to implement it for each transformation-level. Within the second half of the following posting you will find various links to incremental-examples and exists() which is needed to keep the qvd-loadings optimized: Advanced-topics-for-creating-a-qlik-datamodel.

- Marcus

Re: Need suggestions in Data-model

paulwalker — Wed, 13 Feb 2019 08:33:07 GMT

I did incremental load in QVD Generator and working fine.

These concatenate using all QVD's only, Loading process taking hardly 11 mins, but QVD creating more than 40 mins.

Re: Need suggestions in Data-model

marcus_sommer — Wed, 13 Feb 2019 09:12:32 GMT

Are all QVD's loading optimized within the incremental generator-process and by concatenate them? Are all transformations included within the incremental approach?

- Marcus

Re: Need suggestions in Data-model

paulwalker — Wed, 13 Feb 2019 09:28:29 GMT

QVD generator is complete RAW data, no transformations. (fetching RAW data from SQL)

ETL: I did few transformations and concatenated all the QVD's in to one (Finally creating one QVD).

-- Paul

Re: Need suggestions in Data-model

marcus_sommer — Wed, 13 Feb 2019 09:53:16 GMT

It does not really answer the above questions if all (respectively the heavy ones) loadings are optimized and all transformations are included within an incremental load-approach.

Beside this your answer sounds that there is one generator- and one datamodel-application. If it's the case it might be useful to split them into multiple applications because they could then run in parallel (depending on the performance of your database/network and your resources of CPU, RAM and storage and/or the number of available machines). Further not each single-task might need to be updated each time and might be therefore skipped n times.

- Marcus

Re: Need suggestions in Data-model

paulwalker — Wed, 13 Feb 2019 10:01:50 GMT

Hi Experts,

I have attached sample data for 2 tables, please anyone can help on Link table.

-Paul