topic Re: Splitting up QVDs in QlikView

Splitting up QVDs

Zaga_69 — Sat, 16 Nov 2024 03:38:34 GMT

Hi all,

Currently we have big fact table stored in a QVD. In order to improve the App performance we have two approaches:

1. Splitting up the App by using Where statement in the App. That way, the app will take just the relevant data (e.g.

Load * Fact.QVD

Where BusinessLine='Chemicals'+

As a result we will have many apps as many business lines we have.

2. Splitting the Fact.QVD in multiples smaller QVDs (e.g by business line)

FACT_Chemicals.QVD

FACT_PreciousMetal,

That way, each app has their own QVD.

I would like to know your opinion. What would be the best approach to have better performance.

Thanks

Re: Splitting up QVDs

zhadrakas — Wed, 08 May 2019 10:46:12 GMT

if you can handle the QVD size i think you can leave it as it is.
I generally split them at a size bigger than 40GB per QVD.

General Approach for me is always Keep the optimized load and use where exists.
That way i would go like:

REDUCE_DATA:
LOAD * INLINE [
BusinessLine
Chemicals
];

Load *
FROM yourQVD.qvd(qvd)
WHERE exists(BusinessLine)
;

drop table REDUCE_DATA;

Re: Splitting up QVDs

marcus_malinow — Wed, 08 May 2019 10:58:22 GMT

Hi,

the answer to this is dependant on where you are seeing poor performance. Is it that your reloads are taking a long time, or that your application is not responsive when in use?

If the issue is with your reloads, then perhaps instead of splitting by business line, an incremental load based on a date field may be useful.

if the issue is with application performance, then the place to start looking is in your data model and expressions. Do you have a star schema? Are your keys integers?

Marcus

Re: Splitting up QVDs

Zaga_69 — Thu, 09 May 2019 21:39:00 GMT

Hi. Thanks, finally we decided to split up the application in stead of splitting up the QVDs. We conduct the following performance Analysis:
We use dev tool for chrome to calculate how much time need a specific sheet to load in the app. We compare the old version (i.e. the one which contain all rows) vs the new version (i.e. an app that contain a subset of the rows). However, the differences are not so big. For example, in the old app, a sheet in average takes 7 seconds and in the new one takes 6.30 seconds. Any idea, if the amount of rows in a model has a impact in the front-end? thanks!