I am facing a issue, Actually , i have the log file size is around 1.5GB.
If i am trying to load this log file into Qlikview, then getting error "Out of virtual and or/Logical memory allocating820.125 MB" after fetching 22 lakhs records.
My pc has 4GB Ram. Is there any other way to do the reload.
I'm assuming that you are using a 32 bit operating system - if that's true, you can only use 2GB for a single job... You can approach it in two ways - increasig available memory or decreasing the amount of data.
Ways to increase memory:
- if you are on 32 bit OS, you can try using a flag that allows 3GB of RAM to be allocated to a single job (it's somewhere in BOOT.INI, look it up)
- You can try installing a 64 bit OS (not sure how mych trouble would that be for you...)
- If you have access to a server with more memory, you can try running it on the server
Ways to decrease the amount of data:
- examine the structure of your log file - maybe you can reduce the amount of columns that you are loading
- if you are loading the whole string and then parsing it into fields within your script, - you might be able to load your file in portions. Identify date period that could fit into a single load (1 month, 3 months, etc...) and load individual periods, storing the resulting tables in QVD files. At the end, simply load all resulting QVDs into QlikView - you might be able to work around the memory issue this way.
Oleg Troyansky wrote:- if you are on 32 bit OS, you can try using a flag that allows 3GB of RAM to be allocated to a single job (it's somewhere in BOOT.INI, look it up)
As a caution for setting the 3GB flag, on my machine it prevented it from booting. It took our hardware people a couple hours to fix it. I assume this is atypical, but I thought I'd toss it out there.
