App size increased after concatenating qvd

[deep2021]

Hi All,

It has been observed that the application size is increased after qvd concatenation.

Previously it was around 1 gb now it has increased around 10 gb.

I just used simple *.qvd to concatenate all qvds.

In the previous version we used CSV files and there *.CSV was working fine to concatenate table.

Can you please suggest how should I go ahead.

Thanks 

[henrikalmen]

Are you saying that you had for example 10 CSV files, and you loaded them into the application. Then you converted the csv files to QVD instead, and when loading those qvd files you get another result even though the qvd:s have the exact same information as the csv files had?

[deep2021]

Hi,

1. I am fetching data from Rest API.

2. For each date there is a separate QVD.

3. Previously the data was in CSv format and we used the same CSv data in qlik application.

4 although we have a same data sets, it has been observed that, the API data (that we fetched from qvd) is 10 times bigger than the csv data.

After my investigation, i found that record numbers are same and columns are also same.

But still there is increase in app size.

Ca you please suggest on this.

Thanks

[henrikalmen]

You are fetching data from an API, and you are storing the data in QVD? But you used to store the data in CSV? And when loading the data into a qlik sense app, the app size is bigger when you load the exact same data from qvd:s instead of loading it from csv:s?

That doesn't make sense to me. Try creating a new application and just load the qvd data. And create another new application and load the same data från csv:s instead. There shouldn't be a big difference in size between those two apps.

[deep2021]

After loading data from QVD the still the issue is there. There are no differences.

Thanks

[henrikalmen]

Are you absolutely sure that you are loading the exact same data in the same way, and the only difference is that when you load from csv the app file size is smaller than when you load the same data from qvd instead?

[marcus_sommer · Accepted solution]

The same data-set leads to the same app-size. If the app-size differs it means the data-set is different. A factor of 10 in the app-size will be very obvious by looking at the available tables and their number of records and fields within the data-model.

Possible causes may lead to the creation-routines of the csv and the qvd and/or to the way you load the data again - loading per wildcard like *.qvd is a rather bad practice because it will load all qvd-files and not certain ones.

[deep2021]

Hi,

When I compare same QVD`s from different sources I found that there are differences in field length.

Can you please suggest on this. Please find the scenario as per the below,

for instance,

For the below field length and number of records from CSV

Same values from API-

Field Name	Target ONLY Null	Target Null Fields	Field Unique Values	Field Length
A.accountId	No	-	1,258	17,214

Same values from API data

Field Name	Target ONLY Null	Target Null Fields	Field Unique Values	Field Length
A.accountId	No	-	1,218	16,960

[henrikalmen]

So your data is not the same. You have different data in your CSV:s and your QVD:s. I don't know what you mean with the headlines "Same values from API-" compared to "Same values from API data".

[deep2021]

Data is same but sources are different. And there are differences in app size when I fetch the same data from different sources.

In above example there are differences in field length.

Thanks