topic Re: Same data but QVD size increased when pulled from source in QlikView

Same data but QVD size increased when pulled from source

Kushal_Chawda — Fri, 22 Sep 2017 13:02:31 GMT

Hi All,

Below are two scenarios

1) All transformations are performed in QlikView

I have table in source which has around 25 fields and 8 million records.Firstly I created QVD from this table, then I am performing complex transformation in QlikView using this QVD and creating another QVD from QlikView.

2) All transformations are performed at source

Here with the same logic all the transformation are performed at source side and one final table is created in source. I am now pulling this final table into the qlikview and storing it into the QVD.

Issue is that, in both the scenario final QVD created is almost of the same size and records , no. of fields are also same, but surprisingly when I load the QVD created in second scenario in QV application, size of application becomes 1.5 GB, whereas, with the QVD created in scenario 1 application size is 500 MB.

Question:

Both the QVDS are having same no of fields and data, then why there is much difference?

Re: Same data but QVD size increased when pulled from source

marcus_sommer — Fri, 22 Sep 2017 14:23:49 GMT

I assume that the reasons will be in different formattings and/or different field-values. For example if a date-field is really a date and not a timestamp it will be a huge difference in RAM/Storage consumptions. The same is with the format - a pure numeric value has always 8 Byte, a dual-value by a timestamp could have more than 40 Bytes.

Further you will need to check if the numeric fields are really numeric and not string-values and some databases fill values with any kind of chars to a certain lenght.

- Marcus

Re: Same data but QVD size increased when pulled from source

hectorgarcia — Sat, 27 Mar 2021 01:33:55 GMT

check the value you have set to the qlik environment variable allowdatalineage must be set to -1

Re: Same data but QVD size increased when pulled from source

askarkhan — Sat, 27 Mar 2021 03:52:05 GMT

If you are using cross table that will increase your size.

Else try date(datefield,'DD-MM-YYYY') format date during transformation in qlikview. This will give result.