topic Re: Load from Resident table or Disk? in QlikView

Load from Resident table or Disk?

Thu, 07 Jan 2016 06:47:12 GMT

Hello,

Recently I was advise not to use the Resident feature in the Load script and instead use writes and reads from the Disk instead.

I was told that the resident mechanism is less efficient when it comes to large tables.

This fact surprised me as I was sure that to use Resident is best practice as it should make the loads much faster.

Is this fact true? Can someone explain why?

Thanks,

Boris

Re: Load from Resident table or Disk?

manojkulkarni — Thu, 07 Jan 2016 06:58:03 GMT

Refer below link, might help

Non Optimized QVD Load vs Resident Load - qlikview | Qlik Community

Re: Load from Resident table or Disk?

Thu, 07 Jan 2016 07:05:22 GMT

Thanks, looked at the link.

couldn't understand a bottom line.

Re: Load from Resident table or Disk?

Peter_Cammaert — Thu, 07 Jan 2016 07:13:56 GMT

Bottom line is: it depends.

Factors that impact the speed of an optimised QVD load vs Resident Load:

- availability of RAM

- RAM speed

- I/O channel bandtwidth

- HD type and speed

- what you do during this load

- etc.;.

Bottom-bottom-line is: do perfomance tests with a representative amount of data on your particular hardware. Only then make your choice.

Peter

Re: Load from Resident table or Disk?

Thu, 07 Jan 2016 07:18:49 GMT

I have a problem creating a representative amount of data due to several issues. I have big amounts of data but they do not simulate production data (We work with vendors with strict legislation).

Are there rough benchmarks I can use?

Re: Load from Resident table or Disk?

shawn-qv — Thu, 07 Jan 2016 23:06:38 GMT

I'll also mention that if you're reading/writing files across network drives/storage (i.e. NAS, SAN, etc...), that you should also take into consideration the network speed (i.e. 100Mbps or 1Gbps), as this can add a considerable amount of time to your load time.