Incremental load

[Anonymous]

Good day!

Colleagues, i'm new in incremental load and others things connected with it.

Try to describe my need:

In my test model i load all files which have names Test_data*.xlsx from directory.

I need in next load to load only those files, which have today() data modifying.

For expample, if i create one file Test_data4 tomorrow and reload my model tomorrow, it must add to current data (Test_data1-Test_data3) data from Test_data4.

Thanks!

[Anonymous]

Did you check these?

Incremental Load in QlikView – Part1 | Learn QlikView

Incremental Load in QlikView – Part2 | Learn QlikView

[HirisH_V7]

Do check out,

Incremental Load in Qlikview - Sources

-Hirish

[HirisH_V7]

Hi ,

I think for your data sources you can use like this not with incremental load ,

FOR Each File in filelist ('Test_data*.xlsx')


Main_Temp:
  load
  '$(File)' as Name,
FileTime( '$(File)' ) as FileTime,
  Week,
  Sales
FROM
  '$(File)'
(ooxml, embedded labels, table is Sheet1);

NEXT File;


Table:
Load
'$(File)' as Name,
FileTime( '$(File)' ) as FileTime,
  Week,
  Sales

  Resident Main_Temp
  Order By FileTime DESC;
drop table Main_Temp;


NoConcatenate
Excel:
load
  '$(File)' as Name,
FileTime( '$(File)' ) as FileTime,
  Week,
  Sales
FROM
  '$(File)'
(ooxml, embedded labels, table is Sheet1);

Here in this three excel files we can pick the latest file data only by the above code,

PFA,

Hope This Helps,

Hirish

[Anonymous]

Thanks.

Incremental load will be useful for me to reduce time of reload for my model.

In your example table Main_temp makes load of all files from directory..

[Anonymous]

Hi, no..

Thanks for response and info.

I'll read them.

[HirisH_V7]

Yes,

its loads latest table only based on the File time  or else we can go with incremental load like this,

By adding the date field into your excel sheets,

PFA,

Hirish

[Anonymous]

Thanks.

i've read these articles.

But what is main benefit of incremental load?

I thought, it load only latest data and add it to current data of model.

But it load all data -> stores it into qvd's and then add from qvd's only latest data.

I load all files from xls, how i can reduce load time in my situation?