13 Replies Latest reply: Jun 27, 2018 2:46 AM by Alin Dragos RSS

    Operations Monitor Load Failing and taking an average of 35 Min when successful

      I have a new Qlik Sense Server 2.1.1 installation that is only about a week old.

       

      I am seeing CPU spikes and load errors, that as near as I can tell are due to the Operations Monitor Load starting to take a very long time.

       

      From looking at the Operations_Monitor_Reload_Stats_2.1.txt file, it looks like the number of rows it was trying to import got successively larger and the load is completing successfully fewer and fewer times each day, and taking longer and longer to load when it does.  It looks like it is also causing other reload tasks to fail while it is running (which I'm guessing may be creating more log entries and making this problem even worse)

       

      I had some installation challenges based on how I was trying to securely configure this in an AWS VPC, but have those sorted out now.  Is this because of log files generated during this time?  I don't need the historical log files at this point (and would actually rather get them out of the Operations Monitor App.

       

      What's the best way to purge existing log data from the logs and the app?

       

      Also of note, is that the CPU spikes that tend to happen while it is loading start about 30 minutes into the load process.  I upgraded the server to a c4.4xlarge (16 vCPU & 30 GB of RAM), and it still manages to peg all 16 vCPUs.  In the Operations Management app when in the performance sheet, when I try to expand any of the hour rows, it spikes RAM usage and exhausts RAM before coming back and saying Error, Out of calculation memory.

       

      Thanks in advance for any assistance

       

      Performance Summary Pivot.png

       

      RAM Spike when try to expand performance Summary

       

       

      RAM Spike.png

       

       

      Most Recent Operations_Monitor_Reload_Stats_2.1.txt Entries

       

      2992015-10-11 00:07:50INFOFD-QLIK-1Reload StartReloading Operations Monitor 2.1 from FD-QLIK-1 running version 2.1.1+Build:22.origin/release/ms13Operations Monitor
      3002015-10-11 00:08:12INFOFD-QLIK-1Reload FinishReloaded at 2015-10-11 00:08:12 on fd-qlik-1 for 00:00:22 with 94,816 log entries.Operations Monitor
      3012015-10-11 01:07:50INFOFD-QLIK-1Reload StartReloading Operations Monitor 2.1 from FD-QLIK-1 running version 2.1.1+Build:22.origin/release/ms13Operations Monitor
      3022015-10-11 01:09:59INFOFD-QLIK-1Reload FinishReloaded at 2015-10-11 01:09:59 on fd-qlik-1 for 00:02:09 with 142,198 log entries.Operations Monitor
      3032015-10-11 02:07:50INFOFD-QLIK-1Reload StartReloading Operations Monitor 2.1 from FD-QLIK-1 running version 2.1.1+Build:22.origin/release/ms13Operations Monitor
      3042015-10-11 02:13:32INFOFD-QLIK-1Reload FinishReloaded at 2015-10-11 02:13:32 on fd-qlik-1 for 00:05:42 with 232,630 log entries.Operations Monitor
      3052015-10-11 03:07:50INFOFD-QLIK-1Reload StartReloading Operations Monitor 2.1 from FD-QLIK-1 running version 2.1.1+Build:22.origin/release/ms13Operations Monitor
      3062015-10-11 03:19:20INFOFD-QLIK-1Reload FinishReloaded at 2015-10-11 03:19:20 on fd-qlik-1 for 00:11:30 with 360,970 log entries.Operations Monitor
      3072015-10-12 13:07:50INFOFD-QLIK-1Reload StartReloading Operations Monitor 2.1 from FD-QLIK-1 running version 2.1.1+Build:22.origin/release/ms13Operations Monitor
      3082015-10-12 13:43:30INFOFD-QLIK-1Reload FinishReloaded at 2015-10-12 13:43:30 on fd-qlik-1 for 00:35:40 with 881,305 log entries.Operations Monitor
      3092015-10-12 14:07:50INFOFD-QLIK-1Reload StartReloading Operations Monitor 2.1 from FD-QLIK-1 running version 2.1.1+Build:22.origin/release/ms13Operations Monitor
      3102015-10-12 14:43:38INFOFD-QLIK-1Reload FinishReloaded at 2015-10-12 14:43:38 on fd-qlik-1 for 00:35:48 with 882,142 log entries.Operations Monitor