Skip to main content
cancel
Showing results for 
Search instead for 
Did you mean: 
Not applicable

Incremental reload

Hi,

I have a QlikView application which loads the data from a
oracle database...

SELECT * FROM DB."DATE_DIMENSION";

Is there a way to do an incremental update of the data (only reload new data)
without loading the entire data of the db.

How is it possible...?

62 Replies
neetu_singh
Partner - Creator III
Partner - Creator III

Hi,

OK fine, i will check it out because on my side it was working . i will correct it out and then send you again

neetu_singh
Partner - Creator III
Partner - Creator III

Hi Rikab,

I have check that code and now this one is perfectly working. So, please go through with this attached file.

If there is any query then please feel free to mail me.

Hope this one will help you

Bye nd TC

Not applicable
Author

Hi Neetu,

Thanks for your reply. I was running the application but I was not able to find whether incremental load is working fine or not.

What I noticed is it fetched 144rows when I reloaded the application. By question is why it is loading 144 rows again when we are doing the incremental load.

Last row is the row which inserted in to the excel sheet. If the incremental load is working fine it should have loaded only one row.

I have attached the application again please check it again and reply me.

Not applicable
Author

Hey Neetu,

I am looking forward for your reply. Please reply me at the earliest when you are free. Hope I will get some solution soon.

shumailh
Creator III
Creator III

Rikab,

There are different ways of doing Incremental load, asaan tareeqa hai kai.... humain ek loop lagana hota hai magar iss main ek dynamic variable hota hai (for each File in filelist ('$(path)CHT*.dat')), check out some of my simple example:

if required tu.... pehlai ek configuration file hum bana saktai hain.... and define some variable here if you need it i.e. configuration.txt


-------Configuration.txt-----------
let CurrentDay= day(now()-1) ;
let CurrentMonth= 06;
let CurrentYear= 2010 ;
let mn='06Jun10';
let prepath='C:\QlikView';


next step is u need to create a qvw file for incremental load / generate incremental qvd.


$(include=Configuration.txt);
let
path = '$(prepath)' & '\Input\' & '$(mn)' & '\Account\' ;
let Outputpath= '$(prepath)' & '\Output\' & '$(mn)' & '\Account\QVDs\' ;

for each File in filelist ('$(path)CHT*.dat')
let
CurrentFileName = right('$(File)', (len('$(File)') - len('$(path)')) ) ;
let LoadDate = mid('$(CurrentFileName)', 7,2) ;
load
'$(File)' as FileName,
FileTime( '$(File)' ) as FileTime,
FileSize( '$(File)' ) as Size
autogenerate (1);

Txn:
LOAD
@1:3 as TxnCode,
@4:22 as Acc_no,
@46:51 as Purchase_Date,
@52:63 as Destination_Amount,
@64:66 as Destination_Currency_Code,
@67:106 as Source_Amount,
$(LoadDate) as LoadDate
FROM [$(path)$(CurrentFileName)] (ansi, fix, no labels, header is 27, record is line);
Next File ;

store Txn into $(Outputpath)Txn_$(mn).qvd ;
//store CH$(CurrentFileName) into $(Outputpath)CH$(CurrentFileName).qvd ;

drop table Txn;


path variable use to take input file and outputpath is using for output qvd file. iss main hota ya hai kai daily basis pai transaction ki ek new file aa jati hai jisai hum TXN_mn.qvd main daal daitai hain simple. try the above code, i hope it would help u

Shumail



shumailh
Creator III
Creator III

because of some security checks i couldn't download ur file, can u pls send ur attachement to shumail.hussain@fgb.ae. try to send it in qvw format if possible or zip.

Not applicable
Author

As requested I have sent you the document to your email. Hope to receive some solution from you very soon.

shumailh
Creator III
Creator III

You have 5 attachement in the incremental.rar file... can you tell me which one is not working. plus as the area_mst.xls file updated or more data input and when u load the area_mst.xls file then automatically u get the incremental records. what is the issue?

Shumail

Not applicable
Author

Hi Shumail,

I sent you the sample document to you to work on it. Requesting you to work on it and provide me some solution for incremental load.

By the way I am not able to understand how incremental load will happen automatically as said by you.

shumailh
Creator III
Creator III

You need to schedule your Incremantal.qvw file on publisher to load it periodically.

If this is not you want then I think we are miscommunicating here, can u explain what do u mean by not working?

Shumail