Taking more time while writing application to disk

[sudhakar_c]

Hi,

Currently one of our applications is around 14GB in size. Overall to execute the reload for this sales app takes 2.48 hours to 3 hours. When we checked script logs it appears that data processing/transformation is completed fast enough but while saving this app to the central node is taking lot of time. 

what do i need to debug for this issue - e,g, why it takes more time and what is the action to be taken.

I checked the network request-response time during this period - 

Sense Site details:

1 central node

3 Rim node. -

version - Nov2020

[Maria_Halley]

@sudhakar_c

How long is considered a long time? 

A 14 GB app is a very large app. It is not recommended to use apps this big.

You can test how long it takes to copy the file between the 2 servers . But copying the file is not quite the same as what the scheduler is doing.

There is also other factors to take into consideration, like How fast is the disk, is there other file operations at the same time both on the Qlik Server and the File server etc.

[sudhakar_c]

Hi @Maria_Halley - Happy Diwali.
Thanks for the reply.

Earlier the app used to take around 35 minutes to perform the save operation now it takes more than 1 hour.

The app size is large because it has 2 years of granular data which is the business requirement, this is the most critical application in use for more than 6 years (was deployed on Qlik view as well).

I will share the perfmon and operations monitor logs for reference soon.

My observation from the 'Windows Resource Monitor  -> Network Activity' is that, while this task is in execution, another task from the second RIM node also writes the data to the share drive and the bytes received/sec for the other file is 3 times more. So will there be any additional exploration need to be done for network latency or perference related setting to be checked?

[sudhakar_c]

Hi   @Maria_Halley - Uploading DiskActivity and Network activity screenshots during the time. The same pattern was observed for few days. The screenshots are from the central node where the qlik share is located to save the app.

1. On the disk activity screenshot - 2 nd  line is the app for which low write speed is observed

2. GCPLQlikview is the node where the task is getting executed and APPR05 is another RIM node where another task is in execution.