Skip to main content
Announcements
NEW: Seamless Public Data Sharing with Qlik's New Anonymous Access Capability: TELL ME MORE!
cancel
Showing results for 
Search instead for 
Did you mean: 
Not applicable

fastest way to load a table

Hi Friends,

I have two table

Table 1      Table 2

ID              ID

Fname       Fname

Lname       Lname

Now I have to create a new table taking all values from Table1 and  values from Table2 where ID not exist in Table1

What is the fastest way to load a table like this

thanks

6 Replies
adamdavi3s
Master
Master

There was a big blog post on loading data on Rob's site.

I've always used QVDs rather than resident load to be honest and I think this shows that is faster as well

http://qlikviewcookbook.com/2013/09/fastest-method-to-read-maxfield-from-a-qvd/

This thread might also be useful

Where Exists() and resident load

Not applicable
Author

I didnt explain myself correctly - I do load the tables from QVD but I dont have maximum value or date value

and the ID column isn't continuous.

Therefore i can't use the "Delta load" /"Incremental load"

Is there any other way,

Anil_Babu_Samineni

How about this?

Qualify *;

Table1:

Load ID, Fname, Lname from Table1.qvd (qvd);

Unqualify ID;

Concatenate

Qualify *;

Table2:

Load ID, Fname, Lname from Table2.qvd (qvd);

Unqualify ID;


Final_Table:

Noconcatenate

Load * Resident Table1 where not exists (Table2.ID);

Best Anil, When applicable please mark the correct/appropriate replies as "solution" (you can mark up to 3 "solutions". Please LIKE threads if the provided solution is helpful
adamdavi3s
Master
Master

Oh sorry I didn't mean to imply the incremental load would be needed at all, just that loading from QVD with a single 'exists' item is very quick indeed.

So your script would look something really straightforward like like:

Fact:

Load *

from qvd1(qvd);

Load *

from QVD2(qvd)

WHERE NOT EXISTS (ID);


That will just give you one big table with all the data in, AFAIK based on what you have told us

Not applicable
Author

I tried to build like this, I thought there might be a quicker method,

(i load millions of records)

adamdavi3s
Master
Master

Nope via QVD is by the fastest! we load 120million and it takes about 12 seconds *edit* sorry no it is 79seconds */edit* to load and validate (from memory I could be wrong)

Note that if you're using more than one field in the exists then it will be slower.