Problem to STORE QVD

[#1]

Hi Everybody

I have a question about sentences STORE .

I Concatenate three QVD's  (10 , 5 and 4 GB) and created a new QVD. In resulted i hoped a QVD around of 20 GB but i get a QVD of 50 GB. The Scrip is correct and the data too.

This is correct? (50GB)

Thanks!

[#2]

Hi Richard,

Resultant QVD size really depends on nature of your data. Size is dependent on whether your data is loaded as optimized or unoptimized and joins used. If you can explain ur scenario I'll be able to help with some solution.

[#3]

Hi Richard,

Show to us the script please.

[#4]

Are you doing any transformations on the data like adding some new fields or joining the tables.

[#5]

The script is like this...

Thanks a lot

[#6]

Your script looks like clean and I believe the size of qvd will depend on the no of rows.

[#7]

Richard,

There are two suggestions based on what I understood...

1. Try using Joins instead of concatenate: Concatenate is useful to load chunks of data from multiple tables into one resultant when they have same Column names or almost same columns. Search for posts on differences for concatenate and joins
2. If your tables are having values in a Matrix format try to convert it into straight table using CROSSTABLE and observe data size

Hope it helps

[Clever_Anjos]

If your concatenated tables have very different fields, this issue can shows up.

A QVD is a very specialized file, containing:

1) A XML header in plain text, that describes all fields and another information (Creation Date, qvw filename that stored the qvd, the 'lineage' of your data, and so on)

2) A 'Symbol Table' which stores all different simbols that are present at your data, converted to binary and optimized

3) A indexed compressed colection of rows (one to each row of your raw data)

This 'smart' format is the key feature that leads to very impressive load time when you do an optimized load  from a QVD

If you´re original tables have much more columns, this will impact step 3 what can lead to that issue you have noticed

[#9]

Hi Anjos

I worked for each QVD had the same no of fields and the same kind of data

Now when load the QVD's files i do not have this issue

But the result is the same

[Clever_Anjos]

You have done 'by hand' what QlikView did by itself.

QVD´s are not so 1:1 proportion, it´s not assured that twice the data, twice the size, it all depends of data