Unlock a world of possibilities! Login now and discover the exclusive benefits awaiting you.
Ok i had a simple extract script which was:
INFOTABLE:
Load
*;
SQL
SELECT *
FROM infotable
Store INFOTABLE into $(vDetailsQVDDirectory)\INFOTABLE.qvd;
Drop table INFOTABLE;
exit script;
This extract was then used in a load script:
INFOTAB:
LOAD *
FROM
[$(vDetailsQVDDirectory)\INFOTABLE.qvd]
(qvd);
And loaded everything.
I then wanted to split the load for schedule efficiency and archive purposes to two separate extract scripts:
INFOTABLEPY:
Load
*;
SQL
SELECT *
FROM infotable
WHERE YEAR != '2017';
Store INFOTABLEPY into $(vDetailsQVDDirectory)\INFOTABLEPY.qvd;
Drop table INFOTABLEPY;
and
INFOTABLECY:
Load
*;
SQL
SELECT *
FROM infotable
WHERE YEAR = '2017';
Store INFOTABLECY into $(vDetailsQVDDirectory)\INFOTABLECY.qvd;
Drop table INFOTABLECY;
These are then loaded by using a load script with a * to pick them both up:
INFOTAB:
LOAD *
FROM
[$(vDetailsQVDDirectory)\INFOTABLE*.qvd]
(qvd);
However when loaded together the new way has less rows (and missing data) than the old way.
Why is this happening, I am not limiting any data?
Image to help people:
Finally Achieved, Good and then I didn't understand your intend really. Because, You loaded the two time for single QVD. It is may effect wrong in future
Good to see it's working.
Could you maybe check your logs and compare them?
Would be interesting to see the root cause of the difference, i.e. if both versions of your script actually loaded the same files (note the wildcards used in the first version) and the same amount of records for each QVD.
I imagine that the load was trying to JOIN the tables somehow as the load time is back down to normal.
You can check this by looking at the model after the load. Is there 1 single table with all records or are there two tables?
Your script should auto-concatenate the two QVDs into one resident table, assuming both QVD show same number and names of fields.
You can force concatenation of table LOADs using CONCATENATE LOAD prefix.