topic AW:Re: For each..next -loop and out of memory in QlikView

For each..next -loop and out of memory

Fri, 14 Jan 2011 09:14:35 GMT

Hello!

I have 2 databases with identical table with many columns. Then it would be nice to read these 2 tables with for each..next-loop,

because just the connect string is different. Tables have about 14 million rows in both database.

Version is 32-bit 9.00 SP4. (I can't use 64-bit version, no ODBC for 64 bit in this environment)

If I read these tables with "for each..next" -loop. I get out of memory error when just about 9 million rows are readed from the first db.

If I read these tables without loop (I just copy/pasted load statement and edited connect string) then I can read all the rows without any problems. 14 million + 14 million.

Have you also had weird memory hog problems with "for each..next" -loop. Any solutions?

It would be nice to use loop with identical loading phases. but now I can't use it with large tables.

Thank you for your time,

Harri

For each..next -loop and out of memory

sparur — Fri, 14 Jan 2011 10:00:23 GMT

Hello, Harri

why do you need for each..loop?

why don't you want to load one by one table?

For each..next -loop and out of memory

Fri, 14 Jan 2011 10:54:39 GMT

Hi!

I would like to use loop- becaus now we have 2 databases, but in the future 8 databases.

Then it is simplier to manage loading script, because load/select lines are only once in the script.

Easier to maintain etc.

Yours, Harri

AW:Re: For each..next -loop and out of memory

Fri, 14 Jan 2011 11:29:49 GMT

Hi Harri,

I would strongly recommend you to switch to a 64-bit enviroment. QV holds all its data in memory. And with any 32bit-OS there is definitely a RAM-limit (clearly <4GB), depending on different parameters. For details please refer to other threads, you will find many with this topic.

As you told your data will grow rapidly. Let's say to 14*8 Mio rows. You will likely have not only loading issues, but also an issue with your endusers. It can be very uncomfortable when your and/or your endusers demand is growing and your infosystem can't keep up with it.

Regards, Roland

For each..next -loop and out of memory

jjordaan — Wed, 23 May 2012 21:10:19 GMT

Hi Harri,

May I ask how did you created that For Each Next loop especially with the connection string.
I'm trying the same thing. We have a SQL Server and 6 databases that are all the same tables and columns.

I failed to create a loop so that both databases be read.

Thx for your help.