topic Loading QVD give much more records in QlikView

Loading QVD give much more records

francisvandergr — Thu, 24 Mar 2011 19:42:23 GMT

Today i am first working with QVD's. what is my challenge. I want to compare current data with data from the past. Every day i made a qvd file with the data from that moment (i think). So i have some QVD's.

In my source QVW i have 200 records in a table. When i make a qvd of this table and load in it in a new QVW i have 45990 records. Does anybody know how this is possible ? I load one qvd

Loading QVD give much more records

Miguel_Angel_Baeyens — Thu, 24 Mar 2011 20:17:24 GMT

Hello Francis,

Can you post some of your code where you store those QVDs and how you load them? Are you concatenating, joining some tables?

Regards.

Loading QVD give much more records

francisvandergr — Thu, 24 Mar 2011 20:28:46 GMT

Her i have my script. Thanx for your reaction

Loading QVD give much more records

Miguel_Angel_Baeyens — Thu, 24 Mar 2011 20:33:23 GMT

Hello Francis,

I see you do after the binary load a loop to store some tables into QVD files. Is taht loop correct (not the syntax, but the tables it has)? Does the creator script store proper number of records? This is easy to check, create a new file and open just one of the QVDs you have previously created.

In the creator, are you concatenating explicit or implicit or doing any partial reload, or join?

Hope that helps.

Loading QVD give much more records

francisvandergr — Thu, 24 Mar 2011 21:01:07 GMT

Sorry but i think i don't understand you. The qvd's that are created are good. All tables from the source qvw comes in a filestore with the extension QVD.

One file of those QVD's i load in a new file. And then i see that there are much more records. What do you mean with the creator, is that the source from where the QVD's are made ?

Loading QVD give much more records

francisvandergr — Thu, 24 Mar 2011 21:10:01 GMT

i concatenate nothing and join nothing when i load the qvd's. I load one table

Loading QVD give much more records

Miguel_Angel_Baeyens — Thu, 24 Mar 2011 21:11:00 GMT

Francis,

In the text file you attached above, you are doing a binary load, so you are loading all its records into a new document, and this document will have the same tables and records you have in the other document. Then you do a loop to store each of them into a QVD file. Then you do another loop to drop all of them.

My question is about that QVW file you are binary loading, how the data is built in there and if there is a chance that some records are being loaded several times.

At the end of the txt you posted you are loading using wilcards instead the name of the file alone. By itself that's okay, but depending on the name of the files (for example, timestamps), that means you can be loading several files, so there you are the lots of records. Is that possible?

\\FS01\303327$\data\QlikView\Werkkapitaal_Historie\Historie\24*werk*.QV

If you have more than one file that starts with "24" and has "werk" in its name, then you are loading actually more than one file, so it's likely you will find more records that in one file alone.

Sorry if I didn't myself clear before.

Hope that helps.

Loading QVD give much more records

francisvandergr — Thu, 24 Mar 2011 21:26:43 GMT

Hi Miquel,

Thanx for your answer. I have enclosed my script so you can see it. I load one QVD. I understand that there is a possibility that there are more files, but that is not so.

Loading QVD give much more records

Miguel_Angel_Baeyens — Fri, 25 Mar 2011 05:27:22 GMT

Hi Francis,

The file appears to be empty.

Regards.

Loading QVD give much more records

francisvandergr — Fri, 25 Mar 2011 09:35:00 GMT

Miquel,

Here i send you a new file. This is not empty.

francis

Loading QVD give much more records

Miguel_Angel_Baeyens — Fri, 25 Mar 2011 09:40:47 GMT

Yes, now it opens fine.

What tables is giving you trouble when storing/loading showing more records than expected?

Loading QVD give much more records

francisvandergr — Fri, 25 Mar 2011 09:51:33 GMT

table: werkkapitaal

Loading QVD give much more records

Miguel_Angel_Baeyens — Fri, 25 Mar 2011 10:18:57 GMT

Hello Francis,

At a first glance, I see that Werkkapitaal is loaded from DBOP what in turn is loaded from crop_TEMP2 and this one is within a loop, returning 32,7K+ records.

DBOP is concatenated to Prijsverschil and to Crop_NoOrderTemp. Crop_NoOrderTemp is a joint of vwQV_Werkkapitaal_CROP with vwQV_WerkkapitaalCROPNoOrder.

So there are indeed a chain of joins and concatenates that may cause that amount of records in one table. This table is later loaded (through a binary load) into the file that loops the names of the tables and stores them into QVD files.

Are you getting these same results?

If I now do a binary load of this file and store Werkkapitaal into a QVD file, it will store no less than 32000 records.