Reload or Incremental Load?

[sspawar88]

what happen if we schedule reload at server without using incremental load in script?

what is the criteria to use incremental load and Reload ?

New to QlikView‌

New to Qlik Sense

‌App Development‌

[petter]

An incremental load is an incremental read from your source. You will have to combine the "increment" from you source with all or some of the historical data you have stored as QVD-files from previously. Then you will have your complete dataset in QlikView so you do a RELOAD anyway. So the question I think it more precisely posed as FULL (RE)LOAD or INCREMENTAL (RE)LOAD?

The incremental reload/load is only good as long as you have some benefits out of it like:

- Saved time from querying the source.

- Less load on the source.

- Need for keeping a longer and more consistent history in your QlikView environment (Datamart / DWH) than the source can provide you with.

[#3]

Incremental load is recommended (by Kimball methodology) for the largest Fact tables in DW.

Typically only the latest (last 24 hour period) is loaded into the Fact Table (or the last month for monthly load etc).

Dimensions usually are loaded fully (=Slowly Changed Dimension SCD type1).

Thus previous population of the dimension is completely overwritten each time.

Large dimensions may be loaded with incremental population.

In Incremental style population the existing population of the target table may be compare with the source table (and only new rows identified by ETL/ETL tool).

Other approach is "bookmarking mechanism": the last loaded into target row is memorized and only latest  rows (that go after the loaded one) are submitted into ETL. Bookmarking can be done on Datetime column or on identify column (i.e. "monotonically increasing values" are needed).

Hopefully it answers the question.

Alexei Akimov