topic Re: Incremental Load in QlikView

Incremental Load

konidena — Wed, 25 Nov 2020 16:16:04 GMT

Hi Team,

I am doing the incremental load.

First I have 100 rows of data in excel.

EmpNo, Names, Salary

EmpNo is the Unique id to perform my Incremental load.

Question:

Now I have updated Salary of the EmpNo 70, 71, 72 and added two new EmpNo: 101, 102.

When I am doing the addition of this two new Hempen, I am able to do successfully.

But, I am not able to get the updated records of the EmpNo 70, 72, 73

How to perform Incremental Load in this type of scenarios?

Regards

Srinivas

Re: Incremental Load

ashok1203 — Wed, 28 Sep 2016 13:14:35 GMT

Hi Sri nivas, Try to use the below link it might some helpful to you.

https://www.analyticsvidhya.com/blog/2014/09/qlikview-incremental-load/‌

If it is not, let me know.

Re: Incremental Load

konidena — Wed, 28 Sep 2016 13:20:14 GMT

Hi Ashok,

I followed this approach already. It is not helpful.

My scenario is

New file has 102 records. In that, 70, 71, 72 are updated with new salary.

Incremental Load - Update scenario is not working here as expected.

Regards

Srinivas

Re: Incremental Load

marcus_sommer — Wed, 28 Sep 2016 13:24:59 GMT

Take a look here: Slowly Changing Dimensions.

- Marcus

Re: Incremental Load

ashok1203 — Thu, 29 Sep 2016 06:40:14 GMT

Can you provide the sample table structure to understand it clearly.

Re: Incremental Load

Anil_Babu_Samineni — Thu, 29 Sep 2016 06:48:56 GMT

How is your script looking

And have you update the value from excel? If yes, why do you require the incremental load. Would you elaborate more

Because, if you update from excel it should choose one you reload from script. Have you created qvd for before updating?

Re: Incremental Load

shraddha_g — Thu, 29 Sep 2016 09:07:42 GMT

With this data you can add one more column for "last updated on"(Date field).

And then you can use this field in incremental load.

by this you can also track updated as well as newly added records.

I hope this is helpful.

Re: Incremental Load

adamdavi3s — Thu, 29 Sep 2016 09:10:34 GMT

Do you mean that you are adding rows when changing data?

i.e. there are now two rows for 70?

If so then can you just rowno() the import then only bring in the row with the max rowno() for each record?

Re: Incremental Load

konidena — Thu, 29 Sep 2016 09:59:34 GMT

Hi Anil,

First i have extracted the data from excel to qlikview.

Number of rows are 100.

Later, the excel has updated with 2 more new rows and also there is an update in the salary for the records of the Employee ID: 71, 72, 73.

How to get the new info according the new updates

Regards

Srinivas

Re: Incremental Load

adamdavi3s — Thu, 29 Sep 2016 10:03:37 GMT

Why are you not just reloading the spreadsheet to overwrite the data?

Do you need to record the old data?

If so then load the date/time as you load the spreadsheet, then map this to your data

Re: Incremental Load

Anonymous — Thu, 29 Sep 2016 10:46:06 GMT

Hi Sri,

Load first your excel, and then, load your QVD With this condition

WHERE Not(Exists(EmpNo))

Then Save your QVD.

You don't have a date field to compare the update, then, the most simple way is load new data and load old data that don't exists on new data.

Regards!

Re: Incremental Load

ashok1203 — Thu, 29 Sep 2016 10:57:42 GMT

Hope this will work fine.

Re: Incremental Load

ashok1203 — Thu, 29 Sep 2016 10:58:24 GMT

Srinivas did you got the solution ?

Re: Incremental Load

Anonymous — Thu, 29 Sep 2016 11:03:31 GMT

A note on my review, this procedure only works if you haven't loaded previously any field called EmpNo, if you have a loaded previously a field that is so called EmpNo this will not work.

In that case, when the you load the Excel file, you have to create a new field as this

"EmpNo as EmpNoAux"

and the condition when loading the QVD has to be

WHERE Not (Exists (EmpNoAux, EmpNo))

Regards!