Load takes 3 hours. Any faster way to test

[alec1982]

Hi guys,

one of the applications that i have takes 3 hours to load and i am making few changes to few tables. Do you know of a quicker way to test instead of waiting 3 hours?

Thxs,

Alec

[Nicole-Smith · Accepted solution]

You can go into Debug mode and choose a Limited Load to only load in a small number of records:

[#3]

Hi Alec,

to test your model and load syntax use the debugger and load only 10 lines.

BEWARE, don't try to find consistancy in the data just for checking script and model

best regards

Chris

[its_anandrjs]

Instead of checking and loading whole data try with loading few tables in another application and then manipulate your data there.

[#5]

Hi alec,

well, you can either limit the data amount from the data source ( FIRST 100 LOAD* FROM TABLE) or use the partial reload embbeded in qlikview instead of reloadin the whole data.

wishing it'll help you,

regards,

Benjamin Drouin

[roger_stone]

If there are some tables you don't need, insert an EXIT SCRIPT statement before you load them. If the tables are on separate tabs, it helps, as you can move them around and put them after the EXIT SCRIPT.

[alec1982]

Hi,

Can you please explain to me how the partial load helps here.

Thxs,

[#8]

Alec

Limited Load will not take time to load because it loads only 10 first lines of each table and build the model.

So this help to verify script syntax and model without loosing time.

If everything's ok, you can load the whole bench of data in qvw

best rgards

Chris

[MarcoWedel]

Hi Alec,

maybe the additional usage of the sample prefix could also help you by only loading a small portion of your data.

See help:

Sample

The sample prefix to a Load or Select (SQL) statement is used for loading a random sample of records from the data source.

All records will be read but only some of them will be loaded into QlikView.

The syntax is:

sample p ( loadstatement | selectstatement )

where:

p is an arbitrary expression which valuates to a number larger than 0 and lower or equal to 1. The number indicates the probability for a given record to be read.

Note:
It is not possible to use the sample prefix when loading an optimized QVD file.

Examples:

Sample 0.15 Select * from Longtable;

Sample(0.15) Load * from Longtab.csv; (Note: The parenthesis is allowed but not required.)

Although it might not reduce the database load times itself, depending on the amount of transformations your script performs, it might result in an overall execution time benefit because of the reduced amount of data loaded.

An advantage could be, that you still have more associated values in your data model compared to only loading the first x rows of source tables (which might contain less distinct values in sorted fields).

hope this helps

regards

Marco