topic 100%CPU + 50GB RAM Usage in Expression Editor in QlikView

100%CPU + 50GB RAM Usage in Expression Editor

Anonymous — Fri, 28 Apr 2017 17:55:49 GMT

Hi,

I'm experiencing high RAM and CPU usage while using the expression editor in Qlikview x64 version 12.10.020200 SR4.

Any change in the expression leads to 100% single thread CPU usage + 50GB RAM usage for a couple minutes.

Is there any reason for Qlikview to compute the expression while the user makes the changes?

After a while the RAM usage drops to normal levels.

Is thata BUG? Is there anything wrong with my expression?

The SQL query would be as simple as:

SELECT COUNT(A)

FROM TABLE

WHERE B > 2*C

GROUP BY D

Table would be like:

A B C D

1 10 30 A

2 20 25 B

...

Can't Qlikview do a simple select efficiently?

Re: CPU + RAM Usage Spikes in Expression Editor

jonas_rezende — Fri, 28 Apr 2017 19:38:03 GMT

Hi, Bruno Garcia.

Share an image of the data model.

Regards,

Jonas Melo.

Re: CPU + RAM Usage Spikes in Expression Editor

Anonymous — Sat, 29 Apr 2017 03:51:32 GMT

‌Its a single table with a few million lines.

The trouble begins as soon as you insert the equal sign between double quotes.

Re: 100%CPU + 50GB RAM Usage in Expression Editor

Peter_Cammaert — Sat, 29 Apr 2017 13:44:50 GMT

This implies that QlikView somehow tries to precalculate the expression search you are using. Might be a bug.

Should be easy to test if the same happens in 11.20 if I only had a simple table with a few million lines of data...

Re: 100%CPU + 50GB RAM Usage in Expression Editor

Anonymous — Sat, 29 Apr 2017 18:08:31 GMT

Same issue reported here: 40 Mb file exceeds 3 Gigs on Task Manager

It seems Qlikview uses some really crappy algorithm for aggregation...

Time to go back to school guys, relational database technology is almost 50 years old!

Re: 100%CPU + 50GB RAM Usage in Expression Editor

rwunderlich — Sun, 30 Apr 2017 00:51:46 GMT

Interesting. I had not noticed this before. But when the modifier value contains a leading "=" (Advanced Search), QV seems to calculate the expression on the fly as you type. This can be annoying.

I tested on V11.20SR15 (and earlier QV11) and V10Sr6 and the behavior is there as well. A workaround is to type the leading "=" after you have completed your editing.

I've attached an example that demonstrates the issue.

-Rob

http://masterssummit.com

http://qlikviewcookbook.com

Re: 100%CPU + 50GB RAM Usage in Expression Editor

satishkurra — Sun, 30 Apr 2017 17:00:28 GMT

Hi Rob

Could you please help me on this thread.

Apology for spamming this post...

Expression Help on Current Fiscal Quarter

Re: 100%CPU + 50GB RAM Usage in Expression Editor

Anonymous — Mon, 01 May 2017 22:42:48 GMT

Thanks for the feedback, Rob!

I'm going to avoid the memory use problem by moving the calculation of the ration between the two columns to the load script and make the set expression using a constant instead of making reference to two columns in the same row.

But I really don't get why Qlikview needs so much memory in order to do this kind of aggregation, it's many times greater than the original data uses. Is there a better way to do it?

Do you think there is room for improvement in the way Qlikview is implementing this kind of set operation?