Error loading huge amount of data

[#1]

Hi,

I'm trying to load the Apache and IIS logfiles from different servers but Igot an error:

"Out of Virtual and/or logical memory, allocating 2MB"

I've read in previous posts that it could be because of the data. In that case, all data come from text files that I try to import using Qlikview.

I've also increase the virtual memory with no results....

There are 4 million entries / month aprox. from each system and I can't load data from 2 systems.

How can I load and get info from this kind of sources?

Thank you in advance!

[#2]

Hi,

The issue you describe is a typical lack of RAM memory issues. You can try to improve your script in terms f memory usage, meaning loading one table a time, storing in qvd and drop the table afterwards. Once you have all the qvds, try to load them in a consolidated model, if this does not work for you means that you will have to buy some more RAM memory for the server.

Regards.

[#3]

I'm using a 32 bits system with 4GB RAM, so It has the max memory possible...

Is it better to load data from a database than using a text file?

Thanks

[#4]

Hi Qlikview-tester,

the amount of load is independent of your source. Wether RDBMS or textfiles isn't important. QV works fine with very redundant data, in this case the compress-rate may be very high. Do you need every field etc. ?

Ivan told you some good workarounds but I think you will need sooner or later a machine with a 64bit-Windows and the according RAM. Your data is growing fast ervery month.

Regards

Roland