3 Replies Latest reply: Oct 16, 2010 12:28 PM by Roland Kunle RSS

    Error loading huge amount of data


      I'm trying to load the Apache and IIS logfiles from different servers but Igot an error:

      "Out of Virtual and/or logical memory, allocating 2MB"


      I've read in previous posts that it could be because of the data. In that case, all data come from text files that I try to import using Qlikview.

      I've also increase the virtual memory with no results....


      There are 4 million entries / month aprox. from each system and I can't load data from 2 systems.


      How can I load and get info from this kind of sources?


      Thank you in advance!


        • Error loading huge amount of data


          The issue you describe is a typical lack of RAM memory issues. You can try to improve your script in terms f memory usage, meaning loading one table a time, storing in qvd and drop the table afterwards. Once you have all the qvds, try to load them in a consolidated model, if this does not work for you means that you will have to buy some more RAM memory for the server.


          • Error loading huge amount of data

            Hi Qlikview-tester,

            the amount of load is independent of your source. Wether RDBMS or textfiles isn't important. QV works fine with very redundant data, in this case the compress-rate may be very high. Do you need every field etc. ?

            Ivan told you some good workarounds but I think you will need sooner or later a machine with a 64bit-Windows and the according RAM. Your data is growing fast ervery month.