Out of Physical and/or logical memory

[sunil2288]

HI all

I am using qlikview v9,I am incurring an error while handling 2.3millions of data..The error is showing as Out of Physical and/or logical memory.How can i resolve it. The table having four fields.

Comments awaited..

Thanks and Regards,

Sunil

[sunil2288]

my script is like this

LOAD

SQL

SELECT * FROM FullTable; date, tester, sl, model;

[sunil2288]

LOAD

SQL

SELECT *FROM

FullTable;

date,tester,sl,model;

[sunil2288]

HI all

I loaded allth data nut at the time of viewing any data from the table through any chat object it'showing the error as

OUT OF VIRTUAL AND/OR LOGICAL MEMORY, ALLOCATING 128MB

[pover]

Hi Sunil,

Can you try uploading your script again? Nothing appears. There is a link when you post a message that tells you how to post script.

Regards.

[sunil2288]

Hi Karl,

I am again trying to upload the script

[sunil2288]

LOAD date, tester, sl, model; SQL SELECT * FROM FullTable;

[sunil2288]

Hi Karl

Though the table has 4 fields but it has more than 2 millions of data..

And the script is lky dis

LOAD date, tester, sl, model; SQL SELECT * FROM FullTable;

[pover]

Wow. That is as simple as it gets. I wouldn't usually think 2 million rows and 4 columns would take up more than 2 GB to load. I also have a 32-bit laptop with 4GB with Windows 7 and I just tested loading 2.6 million rows and 39 columns from a qvd I have of General Ledger entries. My processor peaks, but the memory stays pretty stable. I saw the QV.exe process peak at about 600 MB. I done no special configuration to my QlikView o laptop.

With this in mind, I would try the same load on a different computer. I once used QlikView with Vista and had no problems, but there were many people who complained about Vista's memory usage. Wade made a valid comment that if all the data is unique then it will use up more memory, but I assume the values in the date and tester column are not all unique.

You should try running your script first like this:

LOAD date, tester, sl, model; SQL SELECT date, tester, sl, model FROM FullTable;

then like this:

LOAD date; SQL SELECT dateFROM FullTable;

then like this:

LOAD date, tester; SQL SELECT date, tester FROM FullTable;

and so on until the error appears. Report back on the results, please.

You aren't running any other high memory usage programs in parallel to QlikView, right? You could try installing the 64 bit version Windows over your 32 bit computer and I've seen that that allows you to take advantage of the 1GB RAM that the 32 bit version Windows doesn't use, but I would leave that as a last option.

Let's see if anybody else has an idea...

Good Luck

[Oleg_Troyansky]

I'd recommend loading a partial amount of data (for example, 10,000 rows), generate Memory Statistics and run them through QlikView Optimizer. You might find some interesting surprises. For example:

1. Date. Some databases are passing full timestamps instead of dates. Timestamps take up much more space, and they are all unique (distinct), because each time-stamp is storing time up to milliseconds.

2. I'm assuming "tester" is the name of a person - most likely nothing interesting there...

3. Don't know what "sl" and "model" mean - any chance those might be blob fields with huge binary arrays of information?

QlikView Optimizer will show you how much space each field takes up.

[rwunderlich]

Are there more than 4 columns in the SQL FullTable? As Karl suggested, I would try specifying the column names, instead of *, in the SELECT.

LOAD date, tester, sl, model;
SQL SELECT date, tester, sl, model FROM FullTable;

You may be bringing back many columns from the DB and the extra columns would occupy memory even though Qlikview will eventually drop them.

-Rob