QVD size doubling when concatenating 2 identical fact tables

[Anonymous]

Dear Friends,

I have 2 Fact tables with equal number of columns coming from 2 different subject areas. I have concatenated both tables and storing the concatenated table as QVD. I observed that size of the QVD is almost 3 times of individual QVD sizes of those 2 facts.

Fact1 QVD Size if I store individually: 32 MB( record count 1.3 million)

Fact 2 QVD Size if I store individually: 168 MB ( 9 Million)

After I concatenated 2 tables , Combined QVD size is 486 MB.

Please suggest how I can reduce the size.

Br,

Chinna

[novonieldeb]

Hi,

Having same numbers of rows and fields and same data added for second time may not mean that it should just double. The size of a file would contain not only data but various other information like association etc. which by default due to the architecture of QV to work more efficiently in in-memory environment, may cause additional size.

However, I am interested to know about why you want to concatenate these exactly identical data? I mean for what requirement.

[Anonymous]

Dear Stefan,

Many thanks for your time. So it means it is allocating the space where ever it is not found value in other table while making concatenate the 2nd fact table. So it means,

if table A has 100 rows with 5 Columns and Table B has 10 rows with 20 Columns(5 Common + 15 extra columns), it will consume the space for those extra 15 columns for the 100 rows of Table A.

In that case if Table A Individual QVD size for 5 Columns with 100 rows is 10 KB, and Table B with 20 COlumns individual fact able size is 5 KB, when we combine both of these 2 QVDs as single QVD, it may comes around  30 KB as 15 extra columns to reserve space for 100 rows.

So it might be happening like above.

Thanks Again.

May I know the below scenario, what if I maintain 2 fact tables with common columns in my data model ??

I know it will create a synthetic key with the common columns. will it cause any performance or data issues??

Br,

Chinna