incremental load for more than 1 file !!!!!!!!!!!

[qliklearnervir]

We are working on 1 scenario where after loading the initial data , we need to load multiple  files but each and every file should load in incremental load manner like 1^st file than 2^nd file ans so on and follow the inser+update scenario.

I have already loaded qvd file and now i just want incremental scenerio ...insert and update ...

but while running below script only insert scenario is working not update.

please suggest how to make the scenario like :

every day file should load with qvd with insert and update and scenario and so on

like 1st file should be Basic_20170315 in qvd than Basic_20170316 with insert and update scenario and need to update the data as desired.

script code given below:

===============================================

for i = 20170315 to 20170317

vFileName = 'Basic_' & $(i);

Basic:

LOAD *

FROM

$(vPath)$(vFileName).xlsx

(ooxml, embedded labels, table is Sheet1);

Concatenate

LOAD *

FROM

$(vPath)Basic.qvd

(qvd)

where not Exists(id);

Store Basic into $(vPath)Basic.qvd(qvd);

next i;

[adamdavi3s]

Your code looks fine, assuming that the spreadsheet contains the new and updated data?

Any new and modified rows will be uploaded, then Qlik will just load any rows which don't exist i.e. are not new or updated.

We need to know more about your data to advise further

[qliklearnervir]

yes...but i want that file should load like:

first it should load 20170315 file and concatenated with QVD and store.

after that 20170316 file and concatenated with QVD and store.

like each file should load separately...

[adamdavi3s]

This should be what your script is doing?

Where is it going wrong?

[qliklearnervir]

no it is not updating the record.

for example i got 20170315 data having id and status and in 20170316 file same id but status changed....

in this scenario this code will not work.....

this code will only append the data instead of update scenario...

i want some more programmatic code to load insert + update for n number of files....

[adamdavi3s]

Hmmm that is strange, in theory this is what is happening

Day1:

ID

1

2

3

store to QVD

Day2:

ID

2

5

6

load id from qvd where not exists

should only load

1

3

then store

1

2

3

5

6

back into qvd

[vinieme12]

if Basic_20170316 has the latest data then Basic_20170314 should be loaded after Basic_20170316

reverse the order in which you are loading the files

Use Do loop Until instead of For loop to reverse the order

[adamdavi3s]

Figured it out, you need a drop table in there

Sorry that took me longer than it should have to work out.

This is a handy script which will also avoid any errors if the qvd doesn't exist

for i = 20170315 to 20170316
vFileName = 'basic_' & $(i);
Basic:
LOAD *
FROM
$(vPath)$(vFileName).xlsx
(ooxml, embedded labels, table is Sheet1);


LET logqvdfile = '$(vPath)Basic.qvd';
LET baseQVDFileSize = FileSize('$(logqvdfile)'); //test the file size


IF baseQVDFileSize > 0 THEN 
Concatenate
LOAD *
FROM
$(vPath)Basic.qvd
(qvd)
where not Exists(id);
ELSE 
ENDIF

Store Basic into $(vPath)Basic.qvd(qvd);

drop table Basic;
 next i;

LOAD *
FROM
$(vPath)Basic.qvd
(qvd);

[Anil_Babu_Samineni]

I am assuming, you are just storing into qvd. May be you need to fetch stored one by help of

load * from Updatedqvdpath;

[adamdavi3s]

This makes no sense and is not helpful