loading Split qvds giving different results to single qvd with same data

[matthewp]

Ok i had a simple extract script which was:

INFOTABLE:

Load

  *;

SQL

SELECT *

FROM infotable

Store  INFOTABLE into $(vDetailsQVDDirectory)\INFOTABLE.qvd;

Drop table INFOTABLE;

exit script;

This extract was then used in a load script:

INFOTAB:

LOAD *

FROM

[$(vDetailsQVDDirectory)\INFOTABLE.qvd]

(qvd);

And loaded everything.

I then wanted to split the load for schedule efficiency and archive purposes to two separate extract scripts:

INFOTABLEPY:

Load

  *;

SQL

SELECT *

FROM infotable

WHERE YEAR != '2017';

Store  INFOTABLEPY into $(vDetailsQVDDirectory)\INFOTABLEPY.qvd;

Drop table INFOTABLEPY;

and

INFOTABLECY:

Load

  *;

SQL

SELECT *

FROM infotable

WHERE YEAR = '2017';

Store  INFOTABLECY into $(vDetailsQVDDirectory)\INFOTABLECY.qvd;

Drop table INFOTABLECY;

These are then loaded by using a load script with a * to pick them both up:

INFOTAB:

LOAD *

FROM

[$(vDetailsQVDDirectory)\INFOTABLE*.qvd]

(qvd);

However when loaded together the new way has less rows (and missing data) than the old way.

Why is this happening, I am not limiting any data?

Image to help people:

[Anil_Babu_Samineni]

Finally Achieved, Good and then I didn't understand your intend really. Because, You loaded the two time for single QVD. It is may effect wrong in future

[swuehl]

Good to see it's working.

Could you maybe check your logs and compare them?

Would be interesting to see the root cause of the difference, i.e. if both versions of your script actually loaded the same files (note the wildcards used in the first version) and the same amount of records for each QVD.

[matthewp]

I imagine that the load was trying to JOIN the tables somehow as the load time is back down to normal.

[swuehl]

You can check this by looking at the model after the load. Is there 1 single table with all records or are there two tables?

Your script should auto-concatenate the two QVDs into one resident table, assuming both QVD show same number and names of fields.

You can force concatenation of table LOADs using CONCATENATE LOAD prefix.