topic Re: Optimization of time reload in QlikView

Optimization of time reload

Thu, 04 Sep 2014 16:26:48 GMT

Hello everyone,

I made some Qlikview report which load data from several Excel files. Files contain large amount of data and Qlikview load them more than 7 minutes. My problem is if I change one number in some Excel file I need to reload all data again and wait new 7 minutes.

What are my opportunitis of optimization loading data and time reduction? Can I ask somehow if only one Excel file changed to load again only him, or is there any incremental load?

Thanks for any help

Re: Optimization of time reload

Thu, 04 Sep 2014 16:39:52 GMT

Not sure if this is the best solution, but something I do in some apps is use a series of variables to control whether I am loading from the source file or from a buffered QVD that was created during the last load.

//First the variables that control which tables I am refreshing (in the below scenario, only Table2 will refresh from source):

SET Load_From_QVDS = 0; //1 = Load ALL Tables from QVD's, 0 = Set by table below

SET Load_Table1 = 1; //1 = Load from QVD, 0 = From Source

SET Load_Table2 = 0; //1 = Load from QVD, 0 = From Source

SET Load_Table3 = 1; //1 = Load from QVD, 0 = From Source

//Then for each table you would have some loading script that looks like:

If Load_From_QVDS = 1 or Load_Table1 = 1 THEN

Table1:

LOAD *

FROM

QVD\Table1.qvd

(qvd);

ELSE

Table1:

LOAD *

FROM

Source\Table1.xlsx

(ooxml, embedded labels, table is Table1);

STORE TRX INTO [QVD\Table1.qvd];

ENDIF

Re: Optimization of time reload

marcus_sommer — Thu, 04 Sep 2014 19:16:18 GMT

You could look here - http://community.qlik.com/search.jspa?q=incremental+load - if an incremental load could be useful - if not because there isn't any real logic which data has changed you could read & store the filetime() from the excel. If the filetime the same how from previous load then you could load from the qvd else from the xls.

- Marcus

Re: Optimization of time reload

Fri, 05 Sep 2014 07:40:23 GMT

Tnx Marcus,

I was thinking on something like that. If I could read file time and if that time is the same from the previous load, i will skip that or load qvd file. I've never done anything like that, but i will try.