topic Out of Virtual memory error in QlikView

Out of Virtual memory error

ipauldur — Sun, 26 Jul 2009 17:55:15 GMT

Hi,

I am facing a issue, Actually , i have the log file size is around 1.5GB.

If i am trying to load this log file into Qlikview, then getting error "Out of virtual and or/Logical memory allocating820.125 MB" after fetching 22 lakhs records.

My pc has 4GB Ram. Is there any other way to do the reload.

Thanks,

Durai.

Out of Virtual memory error

Oleg_Troyansky — Sun, 26 Jul 2009 22:57:45 GMT

I'm assuming that you are using a 32 bit operating system - if that's true, you can only use 2GB for a single job... You can approach it in two ways - increasig available memory or decreasing the amount of data.

Ways to increase memory:

- if you are on 32 bit OS, you can try using a flag that allows 3GB of RAM to be allocated to a single job (it's somewhere in BOOT.INI, look it up)

- You can try installing a 64 bit OS (not sure how mych trouble would that be for you...)

- If you have access to a server with more memory, you can try running it on the server

Ways to decrease the amount of data:

- examine the structure of your log file - maybe you can reduce the amount of columns that you are loading

- if you are loading the whole string and then parsing it into fields within your script, - you might be able to load your file in portions. Identify date period that could fit into a single load (1 month, 3 months, etc...) and load individual periods, storing the resulting tables in QVD files. At the end, simply load all resulting QVDs into QlikView - you might be able to work around the memory issue this way.

good luck,

Oleg

Out of Virtual memory error

johnw — Sun, 26 Jul 2009 23:35:58 GMT


Oleg Troyansky wrote:- if you are on 32 bit OS, you can try using a flag that allows 3GB of RAM to be allocated to a single job (it's somewhere in BOOT.INI, look it up)

As a caution for setting the 3GB flag, on my machine it prevented it from booting. It took our hardware people a couple hours to fix it. I assume this is atypical, but I thought I'd toss it out there.

Out of Virtual memory error

Fri, 05 Mar 2010 17:13:11 GMT

At my company we have the same issue and have reports of machines that do not like this Boot.ini change - did you ever resolve it ?