I want to Insert only Incremental/Changes Records in QVD file. Can I ?

[shantanu73]

Hello Friend's/Guru's,

I want to Insert only Incremental/Changes Records in QVD file. In short I want to update QVD file without dropping QVD file or removing/deleting all

records from QVD file.

This will be useful/helpful went the record is very high say about 1 billion or more. In that case it will be better update the QVD file which will take very

few time to update instead of removing/deleting or dropping QVD file.

I hope I had able to explain scenario.

Thanks in Advance

Shantanu

[Anonymous]

Hi Shantanu,

You sure can, search for incremental load in the QlikCommunity and you will find tons of examples and documentation on how to achieve this.

Cheers,

Johannes

[Gysbert_Wassenaar]

As far as I know you can't add records to a qvd without loading all records first (no direct append). What you can do is store the new records in a new qvd. So you could create a new qvd every day with just the new records for that day. In the qvw with your front end user intergafe you can serially load all of these qvd's in a loop so users can work with the complete set of data.

[shantanu73]

Hello Gwassenaar,

So, u mean to say. If in a day if I insert new record 8 times as average so there will be 8 QVD files in a day, So 360day*8QVD=2880 QVD file will be created and if we multiple with 4 or 5 year it will be do much. I can consume this solution.

It will be great if I get some other solution where I can do everything(insert/update/delete) in one QVD file only.

Thanks & Regards

Shantanu

[suniljain]

Please find following example for increamental load.

Let ThisExecTime = Now( );

QV_Table:

SQL SELECT PrimaryKey, X, Y FROM DB_TABLE

WHERE ModificationTime >= #$(LastExecTime)#

AND ModificationTime < #$(ThisExecTime)#;

Concatenate LOAD PrimaryKey, X, Y FROM File.QVD

WHERE NOT EXISTS(PrimaryKey);

Inner Join SQL SELECT PrimaryKey FROM DB_TABLE;

If ScriptErrorCount = 0 then

STORE QV_Table INTO File.QVD;

Let LastExecTime = ThisExecTime;

End If

[Gysbert_Wassenaar]

No, that's not what I'm saying. You don't create a qvd for each record.

I don't understand your requirements. Do you need your new records added to qlikview in realtime? If so, you have a challenge. That's not the kind of scenarion qlikview fits well. If you only need a refresh of the data on a daily basis then you can create a daily qvd. That only makes sense if you have a lot of new records every day and when an incremental reload using only one qvd would take a very long time.

Also if you have only 8 records per day on average I don't understand how you can end up with a billion rows.

[shantanu73]

I mean not 8 records?

I want to update QVD File on a hourly basis very day.

I will create and run macro 8 times in a day after every 1 hour. Because I want to update QVD files with new/changes records after every 1 hour.

I hope I had able to explain scenario now.

[shantanu73]

No, This not I am searching.

The process of the code which u had given as follows:-

1. It will filter the record from DB_TABLE which is not enter in File.QVD.

2. Then it will peak a record from File.QVD.

3. By using Inner Join it park's the record of DB_TABLE and File.QVD in Memory

4. And then it will recreate the File.QVD.

It is good for when record is few in the table. But it incase of High volume of record's it will take long time. Which is very time consuming.

I want a solution which will take every few time to insert/update the File.QVD with out taking/peaking the record of File.QVD in memory.

I hope I had able to explain my requriment.

Thank

Shantanu

[shantanu73]

Hello Johannes S.

As suggested by U I had refered to the Incremental Load. But However it work in following fashion as follows:

1. It will filter the record from DB_TABLE which is not enter in File.QVD.

2. Then it will peak a record from File.QVD.

3. By using some Join it park's the record of DB_TABLE and File.QVD in Memory

4. And then it will recreate the File.QVD.

It is good for when record is few in the table. But it incase of High volume of record's(About Billion of Record's) it will take long time. Which is very time consuming.

I want a solution which will take every few time to insert/update from Data Source to File.QVD with out taking/peaking/parking the record of File.QVD in memory. After making some type of union join of Datasource and QVD it will create again the same File.QVD which is lot of time consuming

I hope I had able to explain my requriment.

Waiting for your resonse.

Thank

Shantanu

[shantanu73]

Hello Guru's,

Any body can give me support/solution of the above problem?

Waiting for response

Thank in Advance

Shantanu