topic Re: Incremental load in QlikView

Incremental load

Anonymous — Mon, 18 Jan 2016 13:39:51 GMT

Good day!

Colleagues, i'm new in incremental load and others things connected with it.

Try to describe my need:

In my test model i load all files which have names Test_data*.xlsx from directory.

I need in next load to load only those files, which have today() data modifying.

For expample, if i create one file Test_data4 tomorrow and reload my model tomorrow, it must add to current data (Test_data1-Test_data3) data from Test_data4.

Thanks!

Re: Incremental load

Anonymous — Mon, 18 Jan 2016 13:45:29 GMT

Did you check these?

Incremental Load in QlikView – Part1 | Learn QlikView

Incremental Load in QlikView – Part2 | Learn QlikView

Re: Incremental load

HirisH_V7 — Mon, 18 Jan 2016 14:13:46 GMT

Do check out,

Incremental Load in Qlikview - Sources

-Hirish

Re: Incremental load

HirisH_V7 — Mon, 18 Jan 2016 14:26:01 GMT

Hi ,

I think for your data sources you can use like this not with incremental load ,



FOR Each File in filelist ('Test_data*.xlsx')


Main_Temp:
  load
  '$(File)' as Name,
FileTime( '$(File)' ) as FileTime,
  Week,
  Sales
FROM
  '$(File)'
(ooxml, embedded labels, table is Sheet1);

NEXT File;


Table:
Load
'$(File)' as Name,
FileTime( '$(File)' ) as FileTime,
  Week,
  Sales

  Resident Main_Temp
  Order By FileTime DESC;
drop table Main_Temp;


NoConcatenate
Excel:
load
  '$(File)' as Name,
FileTime( '$(File)' ) as FileTime,
  Week,
  Sales
FROM
  '$(File)'
(ooxml, embedded labels, table is Sheet1);

Here in this three excel files we can pick the latest file data only by the above code,

PFA,

Hope This Helps,

Hirish

Re: Incremental load

Anonymous — Mon, 18 Jan 2016 14:40:36 GMT

Thanks.

Incremental load will be useful for me to reduce time of reload for my model.

In your example table Main_temp makes load of all files from directory..

Re: Incremental load

Anonymous — Mon, 18 Jan 2016 14:41:16 GMT

Hi, no..

Thanks for response and info.

I'll read them.

Re: Incremental load

HirisH_V7 — Mon, 18 Jan 2016 15:19:17 GMT

Yes,

its loads latest table only based on the File time or else we can go with incremental load like this,

By adding the date field into your excel sheets,

PFA,

Hirish

Re: Incremental load

Anonymous — Mon, 18 Jan 2016 21:00:47 GMT

Thanks.

i've read these articles.

But what is main benefit of incremental load?

I thought, it load only latest data and add it to current data of model.

But it load all data -> stores it into qvd's and then add from qvd's only latest data.

I load all files from xls, how i can reduce load time in my situation?