Unlock a world of possibilities! Login now and discover the exclusive benefits awaiting you.
Dear community,
Due to the performance issue on massive data and high frequency of data load from database server (reload every 30 minutes),
we would like to use incremental load in every 30minutes, but to avoid losing of any data, we would like to have a full load once per week.
Is it possible to have incremental load and full load in the same document script?
PS: We are talking about the data - 1 million rows with 2Mb per row in 1 month growth,
and keep growing now.
Thanks,
Chanel
Yes possible check the Week days value and according to that per week activate that code on that day suppose every week Sat Evening or any day use. You have to use same script but in if conditions in first If part use every 30 minutes reload and then in second part use per week condition. By this you get the full load per week.
Hope this helps
Thanks for the idea! but could you provide any sample or website link for this method?
so that i can try it out.
Thanks,
Chanel
Hi,
Definitely, it is possible to have a incremental load and full load in the same document script. You just need to handle them with proper condition. For example, if weekday is sunday and time is in between 9am to 10am than run full load script else run incremental load script.
Please note, since this is running every 30min, you may face issue if the full load is not finishing within 30min.
You can search in the community about the incremental loads and use it with the variables.
Ex:-
Let vWeekDay = Weekday(Today( ) );
If $(vWeekDay) = 'Sat'; Then
Incremental Load Script every week logic
Else
Incremental Load Script every 30 minutes logic
End if
Thanks for the important note!