topic Re: Memory consuption by document on engine from API in Integration, Extension & APIs

Memory consuption by document on engine from API

BuTbka — Thu, 26 Jan 2023 11:48:00 GMT

Hi!
This article describe that we can get list of opened documents on node by open https://<ip-address>:4747/healthcheck
Is there any way to get resource consuption by each of them?

Re: Memory consuption by document on engine from API

Marc — Mon, 30 Jan 2023 05:52:50 GMT

https://<ServerAddress>/api/v1/apps/<AppID>/data/metadata/
This API can be used to see how much RAM/CPU was used during the previous reload, this is used in the App Metadata Analyzer to show the size in RAM for a App.

There is no API that exposes a live view of how much RAM/CPU a individual app is currently using within the Engine.

However you could use Telemetry Logging to record the information to log file.
note that the Info log level should be used only during troubleshooting as it can produce very large log files

because of the I/O generated by telemetry logging this will have some level of resource impact, so is generally used to check the performance of a objects within a App as part of a Release cycle.

https://help.qlik.com/en-US/sense-admin/November2022/Subsystems/DeployAdministerQSE/Content/Sense_DeployAdminister/QSEoW/Deploy_QSEoW/telemetry-logging.htm

Re: Memory consuption by document on engine from API

BuTbka — Mon, 30 Jan 2023 06:57:29 GMT

@Marc thanks for reply!
It is very sad. In the QlikView I can find qvb process with large resource consumption in the task manager. After that I can filter the tasks in the QMC by running state and then find qvb process ID in the log.

Re: Memory consuption by document on engine from API

NadiaB — Tue, 31 Jan 2023 17:59:21 GMT

Hi @BuTbka

Telemetry Dashboard is a good option, just make sure the logging level doesn't go beyond Warning, the documentation can be found here:

https://adminplaybook.qlik-poc.com/docs/tooling/telemetry_dashboard.html

Depending of the size of your nodes you could configure the limits to record CPU and RAM consumption, the site in GitHub has some videos that discuss all those things including step by step setup of the tool. The last time I looked at the documentation it seems the installation process is easier, but is worth to look at the videos as basis for the configuration and general considerations are discussed.

Hope it helps.

Re: Memory consuption by document on engine from API

BuTbka — Thu, 02 Feb 2023 19:43:58 GMT

@NadiaB thanks!
We are already using this dashboard. But sometimes it's useless.
For example when someone runs the bad SQL query on a node and it starts consuming all the resources. In this case, we want to find that person, but there is nothing in the log about this request until it completes (or fails). And if the engine eats up all the RAM, the engine service will restart and in this case we will also not find anything about this request in the telemetry log. Correct me if I am wrong.

Re: Memory consuption by document on engine from API

BuTbka — Thu, 09 Feb 2023 16:15:56 GMT

After some experiments and investigations:

For example when someone runs the bad SQL query on a node and it starts consuming all the resources.In this case, we want to find that person, but there is nothing in the log about this request until it completes (or fails).

In this I was right. We need api function to get this information.

And if the engine eats up all the RAM, the engine service will restart and in this case we will also not find anything about this request in the telemetry log.

In this I was wrong. In this case, the information is present in the log.