Data compression difference based on data load techniques

chiragparikh01 — Tue, 23 Jun 2020 14:03:43 GMT

Hi All,

I am loading one qliksense app (data app). When I load data directly from SQL table I am getting more compression and when I load same data using SQL stored procedure I am getting less compression in final data app.

In data model I have one Fact table which contains data from three QVDs and overall rows are more than 170 million. When I load data using SQL stored procedure and using SQL table in both version the QVD size are same for all three QVDs. But when I load these three QVDs into final data app then with table based QVDs I get more compression and with stored procedure based QVDs I am getting less compression.

In both load I am using following code to load data into fact table.

Fact:
LOAD * FROM [lib://AggrQVDs\T_Fact_Table1.qvd](qvd);
LOAD * FROM [lib://AggrQVDs\T_Fact_Table2.qvd](qvd);
LOAD * FROM [lib://AggrQVDs\T_Fact_Table3.qvd](qvd);

The final app size with SQL table based QVD = 2.8 GB and with the same data and the final app size with the SQL stored proc based QVD = 4.9 GB so there is around 2GB compression difference and I am not really able to understand why Qliksense gives more compression for table based QVD? I have compared the XML header information of QVDs for both version and they seems identical.

Because of this issue I need to load data from SQL stored procedure to SQL table and then in Qliksense QVD and obviously that increases the overall time to load data into Qliksense.

Any suggestion or guidance to understand the Qliksense internal of data compression ?? Is there any way to specify high compression with load statement in qliksense?

topic Data compression difference based on data load techniques in App Development

Data compression difference based on data load techniques