topic Re: Incremental load and full load in QlikView

Incremental load and full load

Wed, 09 Jul 2014 06:19:49 GMT

Dear community,

Due to the performance issue on massive data and high frequency of data load from database server (reload every 30 minutes),

we would like to use incremental load in every 30minutes, but to avoid losing of any data, we would like to have a full load once per week.

Is it possible to have incremental load and full load in the same document script?

PS: We are talking about the data - 1 million rows with 2Mb per row in 1 month growth,

and keep growing now.

Thanks,

Chanel

Re: Incremental load and full load

its_anandrjs — Wed, 09 Jul 2014 06:26:21 GMT

Yes possible check the Week days value and according to that per week activate that code on that day suppose every week Sat Evening or any day use. You have to use same script but in if conditions in first If part use every 30 minutes reload and then in second part use per week condition. By this you get the full load per week.

Hope this helps

Re: Incremental load and full load

Wed, 09 Jul 2014 06:31:14 GMT

Thanks for the idea! but could you provide any sample or website link for this method?

so that i can try it out.

Thanks,

Chanel

Re: Incremental load and full load

Anonymous — Wed, 09 Jul 2014 06:38:41 GMT

Hi,

Definitely, it is possible to have a incremental load and full load in the same document script. You just need to handle them with proper condition. For example, if weekday is sunday and time is in between 9am to 10am than run full load script else run incremental load script.

Please note, since this is running every 30min, you may face issue if the full load is not finishing within 30min.

Re: Incremental load and full load

its_anandrjs — Wed, 09 Jul 2014 06:39:33 GMT

You can search in the community about the incremental loads and use it with the variables.

Ex:-

Let vWeekDay = Weekday(Today( ) );

If $(vWeekDay) = 'Sat'; Then

Incremental Load Script every week logic

Else

Incremental Load Script every 30 minutes logic

End if

Re: Incremental load and full load

Wed, 09 Jul 2014 06:44:38 GMT

Thanks for the important note!