Do not input private or sensitive data. View Qlik Privacy & Cookie Policy.
Skip to main content

Announcements
Join us to spark ideas for how to put the latest capabilities into action. Register here!
cancel
Showing results for 
Search instead for 
Did you mean: 
Not applicable

For each..next -loop and out of memory

Hello!

I have 2 databases with identical table with many columns. Then it would be nice to read these 2 tables with for each..next-loop,

because just the connect string is different. Tables have about 14 million rows in both database.

Version is 32-bit 9.00 SP4. (I can't use 64-bit version, no ODBC for 64 bit in this environment)

If I read these tables with "for each..next" -loop. I get out of memory error when just about 9 million rows are readed from the first db.

If I read these tables without loop (I just copy/pasted load statement and edited connect string) then I can read all the rows without any problems. 14 million + 14 million.

Have you also had weird memory hog problems with "for each..next" -loop. Any solutions?

It would be nice to use loop with identical loading phases. but now I can't use it with large tables.

Thank you for your time,

Harri

4 Replies
sparur
Specialist II
Specialist II

Hello, Harri

why do you need for each..loop?

why don't you want to load one by one table?

Not applicable
Author

Hi!

I would like to use loop- becaus now we have 2 databases, but in the future 8 databases.

Then it is simplier to manage loading script, because load/select lines are only once in the script.

Easier to maintain etc.

Yours, Harri

Not applicable
Author

Hi Harri,

I would strongly recommend you to switch to a 64-bit enviroment. QV holds all its data in memory. And with any 32bit-OS there is definitely a RAM-limit (clearly <4GB), depending on different parameters. For details please refer to other threads, you will find many with this topic.

As you told your data will grow rapidly. Let's say to 14*8 Mio rows. You will likely have not only loading issues, but also an issue with your endusers. It can be very uncomfortable when your and/or your endusers demand is growing and your infosystem can't keep up with it.

Regards, Roland

jjordaan
Partner - Specialist
Partner - Specialist

Hi Harri,


May I ask how did you created that For Each Next loop especially with the connection string.
I'm trying the same thing. We have a SQL Server and 6 databases that are all the same tables and columns.


I failed to create a loop so that both databases be read.


Thx for your help.