Skip to main content
Announcements
July 15, NEW Customer Portal: Initial launch will improve how you submit Support Cases. IMPORTANT DETAILS
cancel
Showing results for 
Search instead for 
Did you mean: 
Not applicable

Out of Virtual and/or Logical Memory Error in Load Script

I'm currently working on 9.00 SR3 32bit / WinXP 32bit / 2.5Gb RAM and am loading a script with 16Million+ records. Below is the load script and the approx. records per table. I've seen posts that report a 2Gb limit on 32bit machines. Would this be the case for my load as well? Also if the 32bit version has a limit of 2Gb the what is the limit on in 64bits?

ClientStandardInput: //16M+

SQL SELECT UDA_Key AS [UDA Key],
ClientID AS [Client ID],
ClientProjectID AS [Client Project ID],
CVMVendorID AS [CVM Vendor ID],
SUM(ClientVendorSpend) AS [Spend]
FROM ClientInputData.cvmsa.ClientStandardInput
GROUP BY UDA_Key,
ClientID,
ClientProjectID,
CVMVendorID;

ClientProjectMaster: //2.5K

SQL SELECT ClientID AS [Client ID],
ProjectID AS [Project ID],
EnrichmentType AS [Enrichment Type],
Frequency,
Period_Date AS [Period Date]
FROM ClientInputData.cvmsa.ClientProjectMaster;

ClientMaster: //0.5K

SQL SELECT ClientID AS [Client ID],
ClientName AS [Client Name],
IndustryType AS [Industry Type],
CVMCategory AS [CVM Category]
FROM ClientInputData.cvmsa.ClientMaster
WHERE Delete_Flag IS NULL;

CVMMaster: //2.5M

SQL SELECT [Vendor_ID] AS [CVM Vendor ID], [Name] AS [Vendor Name] FROM CVMMasterScrub.cvmsa.Vendor WHERE [Name] IS NOT NULL;

4 Replies
johnw
Champion III
Champion III

Yes, a single application like QlikView can only address 2GB of memory on a 32 bit Windows machine. If you have 2.5GB of RAM, there's a good chance you won't even get to 2GB for QlikView. There's an operating system tweak that will let it address 3GB, but I forget how you do it, and when I tried it on my own machine it made it unbootable, and it took hours to get my machine back up and running, so use at your own risk even if you find it.

If you have 64 bit QlikView on a 64 bit operating system, the limit is, I believe, 8192 GB (8 terabytes).

On the surface, I wouldn't expect your data to use anything close to 2GB. But I'm also not seeing any joins, and bad joins are typically where my memory requirements suddenly skyrocket. So I'm not sure what's going on in your case.

Not applicable
Author

Thanks for the quick response John. I'm gonna give the 64bit option a try once our production server arrives.

Not applicable
Author

Hi,
in order to address 3gb see the following link: http://www.microsoft.com/whdc/system/platform/server/PAE/PAEmem.mspx

Not applicable
Author


John Witherspoon wrote:If you have 64 bit QlikView on a 64 bit operating system, the limit is, I believe, 8192 GB (8 terabytes).


I've got the error for 4096 MB on my x64 server with 98Gb RAM . I've loaded almost 100 million of records. The qv.exe session captures 10Gb in memory. All the rest memory is free. But why it's getting the error!!!!

And all what it does, it's just LEFT JOIN for one table, which calculates simple IF formulas from itself.

left join ($(vCurrTZPrefix)DateTimePeriod)
load
SessionId,
if($(vCurrTZPrefix)DateTime = $(vCurrTZPrefix)StartDate, 1, 0) as SessionStarted,
if($(vCurrTZPrefix)DateTime > $(vCurrTZPrefix)StartDate, 1, 0) as ActiveSession
resident $(vCurrTZPrefix)DateTimePeriod;