Application size has increased after loading data from Rest API

[deep2021]

Hi All,

I have a issue as per the below.

1. I am fetching data from Rest API.

2. For each date there is a separate QVD.

3. Previously the data was in CSv format and we used the same CSv data in qlik application.

4 although we have a same data sets, it has been observed that, the API data (that we fetched from qvd) is 10 times bigger than the csv data.

After my investigation, i found that record numbers are same and columns are also same.

But still there is increase in app size.

My previous app size was 1 GB but now it has increase to 10 GB.

Ca you please suggest on this.

Thanks

[Fernando_Fabregas]

Hi ! Perhaps the problem is the empty values and how each interface sends it.

It may be that Qlik is receiving null values from CSV and empty strings ('') from the REST API.

You can try transforming each empty REST string to Null(), with whatever method you prefer, i.e.: 

LOAD if (len(trim(YourField)) > 0, YourField, NULL()) as YourField 

FROM yoursource;

Regards

[deep2021]

Hi Fernando,

Thanks for your help.

After applying above conditions on each and every columns the record counts are reduced drastically. As a result app size is also reduced.

But still there are differences in app size as compared to the original app. Can you please suggest what other parameters do I need to consider while applying transformations?

Is there any parameters that I need to pass in API connection string.

Currently, I am passing the below parameters in qlik sense same as Postman.

Thanks

[Fernando_Fabregas]

Hi! Another tip is to limit useless decimal places in numbers fields (ie 2 places for money fields).

I think Qlik only compresses data in your model for the same values, different numbers can't save space.

To do that, use Round(), Floor() or Ceil().

Regards, Fernando