topic Re: Qlikview question in QlikView

Qlikview question

kushalthakral — Wed, 25 Nov 2020 16:16:04 GMT

Hi All

Recently i have been asked question like some one has done setting in their server that QV application will through timeout error after 60 minutes and an application which is having huge data sets and they dont want to do incremental load as data in tables changes so we cant really use incremental load

so ho can we achieve this ?

Any suggestions on this are welcome

Thanks

Kushal

Re: Qlikview question

marksouzacosta — Mon, 11 Feb 2019 02:26:45 GMT

I'm not sure if I got this right and it could be a very broad subject.
Are you talking about getting a considerable amount of data and save it into a QVD file under 60min without incremental load?

Re: Qlikview question

pradosh_thakur — Mon, 11 Feb 2019 05:16:56 GMT

If this is a interview question then I am not sure what is the context this is asked. But if you have a huge volumes of data, the time out can happen.

Coming to the "not able to do incremental load because the table changes part", incremental load can update and delete record. There is something called direct query that can be used to fetch data directly from the data base but i would not advise that.

Thanks

Pradosh

Re: Qlikview question

marksouzacosta — Mon, 11 Feb 2019 05:26:27 GMT

I have nightmares with Direct Query.
Good advice Pradosh lol!

Re: Qlikview question

kushalthakral — Mon, 11 Feb 2019 07:20:02 GMT

Hi Mark
Yes

Re: Qlikview question

marksouzacosta — Mon, 11 Feb 2019 15:03:24 GMT

I guess one possible solution is doing data partition and reloading the whole set of partitioned data over time but like @pradosh_thakur said, incremental load techniques are quite flexible and you can implement it in almost every single scenario - you can even combine partitioned data with incremental load techniques.

By data partition I mean, in some cases, you can aggregate your data by a field or fields. The most common case is a Date field. So you can save your data like this:
By Year
Sales_2010.qvd
Sales_2011.qvd
.....
Sales_2019.qvd

Or

By YearMonth
Sales_201001.qvd
Sales_201002.qvd
.....
Sales_201902.qvd

Or

By Category/YearMonth
Sales_Local_201001.qvd
Sales_Local_201002.qvd
.....
Sales_Local_201902.qvd
Sales_International_201902.qvd

This way, when you detect that something changed in your data, you can isolate which QVD to perform a full reload or incremental load - but again, that depends on your data structure and how it evolves over time.

Regards,
Mark Costa