topic Re: List boxes with 30M unique entries makes Qlikview crawl in QlikView

List boxes with 30M unique entries makes Qlikview crawl

johanfo2 — Thu, 14 Jul 2011 21:46:28 GMT

So far I have quite good experience with Qlikview. However, I feel now I have ran into a serious issue with large list boxes.

Take this simple script:

Load

RecNo() as A, round(rand()*100000000) as B

autogenerate(30000000);

It generated 30 million rows. First column is a pure counter, and the second are random figures between 0 and 100 Million.

Now, If I create two listboxes of the two, and start to search for a number in column B, the system grinds to a holdt as soon as I start typing. It seems as the system tries to search for each keystroke, and freezes between the. Very annoyong indeed, though it eventually recovers perhaps 10 seconds.

Why do I need this? Well, in my real application I have 30M rows. One of the columns of the row is a unique transaction number of approximately 10 digits, and I want to search this occationally.

My questions are as follows:

* Am I pushing the boundries of Qlikview? Is this normal beaviour?

* How should I otherwise solve this problem?

I'm running this on a i7-2600k with plenty of ram

List boxes with 30M unique entries makes Qlikview crawl

magavi_framsteg — Thu, 14 Jul 2011 21:52:55 GMT

Hi Johanfo2.

Which version of QV are you using?

I have never seen this when I have plenty of RAM and CPU available.

I've only seen this on very slow machines.

Re: List boxes with 30M unique entries makes Qlikview crawl

johanfo2 — Thu, 14 Jul 2011 22:21:54 GMT

Both Qlikview 11 Beta, Personal Edition (64 bit) and Qv 10 SR3 (64 bit).

I just tried it on QlikView 10 SR3:

* Ran the script above

* Added two listboxes (A,B)

* Clicked on the frame of B and wrote '33456' (or whatever number)

Result: Total freeze, which recovers after 15 second. This is on a 4-core i7-2600K with 8 GB ram.

I believe everybody will get this problem, regardless hardware.

List boxes with 30M unique entries makes Qlikview crawl

johanfo2 — Fri, 15 Jul 2011 21:52:32 GMT

I'll add my own findings. It seems as, since column "B" is a number, and my search (F3) enters a string-of-numbers, the system tries to convert all the numbers to strings before comparing. If I on load wite:

Load

RecNo() as A, Text(round(rand()*100000000)) as B

autogenerate(30000000);

So that column B is text, not numbers, then the system doesn't hang on search! Not a very easy "bug" to see.