OUT OF VIRTUAL AND/OR LOGICAL MEMORY, allocating 2MB

[#1]

CROSSTABLE(YM, Sales, 30)
LOAD PKAU_COD, PKAU_DES, PKAU_RES,
AUDI_COD, PACK_COD, CUF, PACK_LCH, PACK_MTH, SUF,
APP3_COD, APP2_COD, APP1_COD, ATC4_COD, ATC3_COD,
ATC2_COD, ATC1_COD, EDA1_COD, STGH_COD, PROD_COD,
PROD_ATC, PROD_LCH, PROD_MTH, MANU_COD, CORP_COD,
MNFL_COD, EDAD_COD, GENE_COD, MARK_COD, CMPS_COD,
NMOLECPS, MTH00LC, MTH01LC, MTH02LC, MTH03LC, MTH04LC,
MTH05LC, MTH06LC, MTH07LC, MTH08LC, MTH09LC, MTH10LC,
MTH11LC, MTH12LC, MTH13LC, MTH14LC, MTH15LC, MTH16LC,
MTH17LC, MTH18LC, MTH19LC, MTH20LC, MTH21LC, MTH22LC,
MTH23LC, MTH24LC, MTH25LC, MTH26LC, MTH27LC, MTH28LC,
MTH29LC, MTH30LC, MTH31LC, MTH32LC, MTH33LC, MTH34LC,
MTH35LC, MTH36LC, MTH37LC, MTH38LC, MTH39LC, MTH40LC,
MTH41LC, MTH42LC, MTH43LC, MTH44LC, MTH45LC, MTH46LC,
MTH47LC, MTH48LC, MTH49LC, MTH50LC, MTH51LC, MTH52LC,
MTH53LC, MTH54LC, MTH55LC, MTH56LC, MTH57LC, MTH58LC,
MTH59LC, MTH00UN, MTH01UN, MTH02UN, MTH03UN, MTH04UN,
MTH05UN, MTH06UN, MTH07UN, MTH08UN, MTH09UN, MTH10UN,
MTH11UN, MTH12UN, MTH13UN, MTH14UN, MTH15UN, MTH16UN,
MTH17UN, MTH18UN, MTH19UN, MTH20UN, MTH21UN, MTH22UN,
MTH23UN, MTH24UN, MTH25UN, MTH26UN, MTH27UN, MTH28UN,
MTH29UN, MTH30UN, MTH31UN, MTH32UN, MTH33UN, MTH34UN,
MTH35UN, MTH36UN, MTH37UN, MTH38UN, MTH39UN, MTH40UN,
MTH41UN, MTH42UN, MTH43UN, MTH44UN, MTH45UN, MTH46UN,
MTH47UN, MTH48UN, MTH49UN, MTH50UN, MTH51UN, MTH52UN,
MTH53UN, MTH54UN, MTH55UN, MTH56UN, MTH57UN, MTH58UN,
MTH59UN
FROM C:\template\PKAU.csv (ansi, txt, delimiter is ',', embedded labels, msq);

(The column from MTH00LC to MTH59UN are all period column. For example, MTH00LC means 201012 )

When i reload the script above, the file always get the error message as "OUT OF VIRTUAL AND/OR LOGICAL MEMORY, allocating 2MB". i wonder it may because of the crosstable.

Can anyone help with this issue? Many thanks!

[peter_turner]

Hello RichaleLY,

That message is all about free RAM on your machine, either not enough of it on the PC to start with or the data your loading is too big for your PC.

Best way to check is to open your windows task manager and watch the graphs with the ram usage, i expect when you reload your QlikView document you'll see a huge spike in ram use until QlikView runs out.

To get around this you could try only loading the first 10? rows of the data to make sure the script is ok, then move the QlikView to a higher spec machine if needed.

Thanks,
Peter.

[#3]

Hello Peter,

Thank you for your reply.

First, i monitored the memory when reloading. The Memory is 2.83 GB and the data has been loaded till 31,449,308 lines when crash, while my PC RAM is 4GB.

Second, i try to cut down some columns and changed script as :

CROSSTABLE(YM, Sales, 30)
LOAD PKAU_COD, PKAU_DES, PKAU_RES,
AUDI_COD, PACK_COD, CUF, PACK_LCH, PACK_MTH, SUF,
APP3_COD, APP2_COD, APP1_COD, ATC4_COD, ATC3_COD,
ATC2_COD, ATC1_COD, EDA1_COD, STGH_COD, PROD_COD,
PROD_ATC, PROD_LCH, PROD_MTH, MANU_COD, CORP_COD,
MNFL_COD, EDAD_COD, GENE_COD, MARK_COD, CMPS_COD,
NMOLECPS, MTH00LC, MTH01LC, MTH02LC, MTH03LC, MTH04LC,
MTH05LC, MTH06LC, MTH07LC, MTH08LC, MTH09LC, MTH10LC,
MTH11LC, MTH12LC, MTH13LC, MTH14LC, MTH15LC, MTH16LC,
MTH17LC, MTH18LC, MTH19LC, MTH20LC, MTH21LC, MTH22LC,
MTH23LC, MTH24LC, MTH25LC, MTH26LC, MTH27LC, MTH28LC,
MTH29LC, MTH30LC, MTH31LC, MTH32LC, MTH33LC, MTH34LC,
MTH35LC, MTH00UN, MTH01UN, MTH02UN, MTH03UN, MTH04UN,
MTH05UN, MTH06UN, MTH07UN, MTH08UN, MTH09UN, MTH10UN,
MTH11UN, MTH12UN, MTH13UN, MTH14UN, MTH15UN, MTH16UN,
MTH17UN, MTH18UN, MTH19UN, MTH20UN, MTH21UN, MTH22UN,
MTH23UN, MTH24UN, MTH25UN, MTH26UN, MTH27UN, MTH28UN,
MTH29UN, MTH30UN, MTH31UN, MTH32UN, MTH33UN, MTH34UN,
MTH35UN

FROM C:\template\PKAU.csv (ansi, txt, delimiter is ',', embedded labels, msq);

The source data has no change. Then, QV can run successfully. To do this, i can only keep 3 years data, but my requirement is to see 5 years data, which means i have to run from MTH00LC(UN) to MTH59LC(UN), not MTH35LC(UN).

Third, i reduced source data to 10 lines, QV can run successfuly.

The problem is that i can neither cut down the columns of the script or reduce the source data, what can i do now?

[peter_turner]

Normal 0 false false false EN-GB X-NONE X-NONE

Hello Richale,

It's defiantly the amount of RAM in your PC.

There is a limit to the amount of RAM an application can use in windows 32bit systems and it sounds like your reaching it at 2.83gb used.

If possible you could try using a server or 64bit PC to process your data in the QlikView document?

The crosstable function is quite memory hungry so you could try 2 things:

1. Make the crosstable function one of the first things your document does, as it should have more free ram to use than later in the document when QV has loaded other data.

You could also try to remove empty fields and replace them with 'Null' values, as this is more efficient for QV.

2. (Best method)

Split your document into 2 parts.

Part 1 is a simple QlikView document that only does the crosstable function and save's the results -either as a QVD file or inside the document as normal.

If you try this and still run out of ram then you might have to look at reducing the amount of fields or date range of data, or doing the crosstable as 2 separate operations.

Part 2 would load the finished data from Part1 and then continue with your script as normal.

Thanks,
Peter.

[#5]

Hello Peter,

Thank you so much for your help.

I'm trying your advice now.