fastest way to load a table

[#1]

Hi Friends,

I have two table

Table 1      Table 2

ID              ID

Fname       Fname

Lname       Lname

Now I have to create a new table taking all values from Table1 and  values from Table2 where ID not exist in Table1

What is the fastest way to load a table like this

thanks

[adamdavi3s]

There was a big blog post on loading data on Rob's site.

I've always used QVDs rather than resident load to be honest and I think this shows that is faster as well

http://qlikviewcookbook.com/2013/09/fastest-method-to-read-maxfield-from-a-qvd/

This thread might also be useful

Where Exists() and resident load

[#3]

I didnt explain myself correctly - I do load the tables from QVD but I dont have maximum value or date value

and the ID column isn't continuous.

Therefore i can't use the "Delta load" /"Incremental load"

Is there any other way,

[Anil_Babu_Samineni]

How about this?

Qualify *;

Table1:

Load ID, Fname, Lname from Table1.qvd (qvd);

Unqualify ID;

Concatenate

Qualify *;

Table2:

Load ID, Fname, Lname from Table2.qvd (qvd);

Unqualify ID;

Final_Table:

Noconcatenate

Load * Resident Table1 where not exists (Table2.ID);

[adamdavi3s]

Oh sorry I didn't mean to imply the incremental load would be needed at all, just that loading from QVD with a single 'exists' item is very quick indeed.

So your script would look something really straightforward like like:

Fact:

Load *

from qvd1(qvd);

Load *

from QVD2(qvd)

WHERE NOT EXISTS (ID);

That will just give you one big table with all the data in, AFAIK based on what you have told us

[#6]

I tried to build like this, I thought there might be a quicker method,

(i load millions of records)

[adamdavi3s]

Nope via QVD is by the fastest! we load 120million and it takes about 12 seconds *edit* sorry no it is 79seconds */edit* to load and validate (from memory I could be wrong)

Note that if you're using more than one field in the exists then it will be slower.