topic Re: QMC Task Error: "No slave scheduler found" in Management & Governance

QMC Task Error: "No slave scheduler found"

daniel_dalnekoff — Tue, 12 Jun 2018 18:45:42 GMT

Hi,

We are getting an error in the QMC when scheduled tasks are starting to queue up (waiting on what is executing to begin). My timeout is set to 120 minutes, but these are immediately failing with the error above (No slave scheduler found).

We have a single scheduler node configured as Master and Slave, and the engine service is running on this node. Confirmed QMC shows the service is running (and so does Windows). Resources on the machine are not maxed out. I am setup to allow 6 concurrent tasks to run (based on 8 cores). It seems like as soon as several tasks start to get in the triggered or queued status this happens.

Does anyone have any thoughts on how to troubleshoot?

Re: QMC Task Error: "No slave scheduler found"

aki_martiskaine — Fri, 13 Jul 2018 05:50:51 GMT

Hello Dan

We're experiencing exactly the same situation for a few days now. Only difference is that we have a 60min timeout, otherwise our setup is the same: single node Master and Slave, services ok, 6 concurrent tasks allowed and server performance isn't maxed.

As soon as task is triggered to start, QMC states: "no slave scheduler found" and the task will move to "Error" state and not retry.

Were you able to get help to your problem?

Re: QMC Task Error: "No slave scheduler found"

daniel_dalnekoff — Tue, 17 Jul 2018 00:35:32 GMT

Aki - We ended up restarting the machines in our environment, and everything returned to normal. It seems like on occasion we run into issues like this (miscellaneous errors with the services) and a restart seems to resolve it. Unfortunately, we can't find any information on troubleshooting or understanding the root cause of the issue.

Re: QMC Task Error: "No slave scheduler found"

Thiago_Justen_ — Tue, 17 Jul 2018 02:18:38 GMT

Here is a quick solution:

Re: Qlik Sense Enterprise Server Tasks has "No slave scheduler found" error

Re: QMC Task Error: "No slave scheduler found"

daniel_dalnekoff — Tue, 17 Jul 2018 04:35:41 GMT

Thiago - Good find - we had tried a similar approach as this with restarting selected services on the scheduler node, but unfortunately for our environment it did not resolve the issue.

Re: QMC Task Error: "No slave scheduler found"

BalaBhaskar_Qlik — Tue, 17 Jul 2018 13:47:27 GMT

May be try this:

Solution is that the Engine must be Enabled on the Scheduler node, otherwise it will not be able to run tasks.

if you are setting up Qlik Sense in a multi-node configuration, and something's not working properly, CHECK YOUR PORTS FIRST!!!! Even if you are sure you got them all opened the way you're supposed to, go back and check your ports against the documentation, every time! It should be the first step in your troubleshooting, especially if you have an extra layer of bureaucracy in having sys admins who are in charge of managing those ports.

Try these:

1. Central node should be master only. and slave or scheduler node should be slave.

2. After distributing the certificate, make sure you restart all qliksense services on both the nodes at least twice.

3. The most important thing, you may get a msg saying "Scheduler has stopped", Then you have to wait for some time(it may be in hours). it is the time which repository takes to get in sync with each other. That's what most people does not do.

4. I will request you to do this in off peak hours.

Change config Fly only master yukon only Slave and start task

Attempting to start task Reload task of big data

Sending task Reload task of big data to slavenode yukon.rbcgrp.com (Slavenode id: 410304ca-4f37-4db2-b411-faf3bcd8e123)

Task completed successfully .

Re: QMC Task Error: "No slave scheduler found"

BeeGees — Mon, 23 Jul 2018 07:52:12 GMT

I have the same issue. Started more or less at the same time. I've upgraded to Qlik June 2018 recently and I'm blaming this one for those. Full machine restart also helps. But after services stop & start due to daily backup I'm starting to face those "No slave scheduler found".

I have central node + rim node. Oryginally i had scheduler running only on central one, configured as master + slave. But I've checked configuration with rim node as master scheduler and central node as slave scheduler (can't do otherwise, since odbc are physically configured only on central node).

Ports are fine - also tested on firewall down.

The only difference is that when Engine service start for the first time i got this error in event viewer:

"Engine GetQrsSettings: Failed to retrieve QrsSettings, status code=404"

But after next startup all works fine.

Also, in task at QMC - I'm able to start them manually with couple retries. It seems that when too much tasks are queued at once some of them are failing with "No slave scheduler found". Max concurent reloads is set to 28 on both schedulers. I'm using half of that at most.

Re: QMC Task Error: "No slave scheduler found"

aki_martiskaine — Mon, 23 Jul 2018 18:50:47 GMT

Hi Dan

Yeah we were able to fix this issue with just restarting all Qlik services. When we looked at the services initially everything was in "Running" state, so no hint there that anything was wrong. But as we've learned, Qlik services seem to sometimes "malfuntion" and still be "Running" in the systems eyes so no flags are raised. I think that was the case with us.

Re: QMC Task Error: "No slave scheduler found"

asparagus — Mon, 13 Aug 2018 06:34:52 GMT

I have the same issue since aprox. 2 weeks.

This morning I recognized that the server has been rebooted this night (I guess by out Windows Admin). The Engine and the Scheduler Service directly crasht after they where started automatically. I tried to stop and start all Services manually in the right order and the Engine Service crashed again. I tried to restart only the Engine service and then it kept running finde.

The Windows Event log says: Engine GetQrsSettings: Failed to retrieve QrsSettings, status code=404

My QlikSense is running on June 2018 release, but when the error first appeared it was runnin gon Feb. 2018.

Re: QMC Task Error: "No slave scheduler found"

RickWild64 — Tue, 21 Aug 2018 11:50:04 GMT

Hi,

Mine started working when I activated all the services on the rim node in the Nodes page of the QMC. We already had QRS, QES and QSS activated, and all the services were running on the rim node anyway. I haven't worked out yet which one did the trick.

Good luck,

Rick

Re: QMC Task Error: "No slave scheduler found"

plexpro52 — Tue, 04 Sep 2018 16:01:00 GMT

Hello Dan et al.,

Our environment is similar to yours, a single node, master and slave, etc. I have a daily task which loads a number of qvd files, and then five app loads chained to its success. A week or two after installing the June release, it started to behave abnormally, three of the five (not predictable as to which three) would fail with the error that you have noted.

Qlik provided us with an idea to try, which worked for us.

They had us add a line to the C:\ProgramData\Qlik\Sense\Engine\settings.ini file. In the [Settings 7] section, we added "DisableNewRowApplicator=0", and that resolved the problem, at least on first execution of the job stream. We'll see if it sticks.

René

Re: QMC Task Error: "No slave scheduler found"

BeeGees — Wed, 05 Sep 2018 07:18:34 GMT

Thx for share. Seems worth to try it.

My workaround was harsh and simple. Since tasks failed randomly the solution was just to automate their retry. Therefore i wrote an SQL query which allows me to define those tasks that failed:

SELECT

"ReloadTasks"."ID",

"ReloadTasks"."Name"

FROM

public."ExecutionResults"

join public."ReloadTaskOperationals" on "ReloadTaskOperationals"."LastExecutionResult_ID" = "ExecutionResults"."ID"

join public."ReloadTasks" on "ReloadTasks"."ID" = "ExecutionResults"."TaskID"

where

"Status" = 11

order by "ExecutionResults"."StartTime" asc

Then I just rerun the task using 3rd party tool QlikSenseTask.exe Works as a charm.

Above is scheduled to repeat every 1 min. I realise that could be possible done using API, but it was faster for me that way.

Re: QMC Task Error: "No slave scheduler found"

Daniele_Purrone — Wed, 12 Sep 2018 08:18:40 GMT

Hi René,

thanks for informing all.
May I ask you who you talked with at Qlik? Was this a support case? We are trying to investigate if this is caused by a bug.

Thanks!

Daniele

Re: QMC Task Error: "No slave scheduler found"

plexpro52 — Wed, 12 Sep 2018 14:22:49 GMT

Hello Daniele,

>> May I ask you who you talked with at Qlik?

Analytics8 is our level 1 support. They worked with the Qlik support organization, who provided this solution.

>> Was this a support case?

Yes, it was a support case with Analytics8. I don't know how Qlik handled it at their end.

René

Re: QMC Task Error: "No slave scheduler found"

Daniele_Purrone — Thu, 13 Sep 2018 08:55:04 GMT

Thanks René, I've found the support case.

We are currently investigating why this happens.
In your situation it was easy to find a workaround, because of the presence of AAALR warnings in the logs.
We tried to apply to same workaround to other customers, and it worked in some cases (even without AAALR warnings being present), it didn't in some other.
The issue might be connected to high cpu usage.

As said, Support is currently investigating this. Anyone having the problem, please try the workaround first, and should not that help, feel free to reach out to Qlik Support (either directly or via the responsible partner, if you have one).

Thanks in advance!

Re: QMC Task Error: "No slave scheduler found"

peter_turner — Thu, 13 Sep 2018 11:08:15 GMT

Hello,

I support a number of customers and have encounted this exact behaviour on at least three seperate installations.

I'll try and list the findings i have so far, and if i manage to resolve the problems...

Servers are running either April or June 2018 (inital release and Patch1) QS, some single and some multinode.

The problem is the best kind, intermittent.

It appears on the QMC task as 'Failed' status, with the information 'No slave scheduler found'.

Tasks will then run fine when manually triggered.

To me it seems to happen more when there are multiple tasks dependent on another task, maybe an issue with the tasks all being triggered together and not being able to wait afew seconds for an engine to become available?

In all cases the servers have lots of available RAM/CPU, scheduler service is running fine as reported by the OS and QMC.

The servers are not overloaded with Tasks.

I have tried adjusting the max concurrent reloads up and down, no change.

On the multinode servers i have adjusted the scheduler service role (between Master and Slave, Master only and Slave only) no change.

Restarting the Qlik Scheduler Service will resolve the issue for a while, depending on the frequency of reloads.

Re: QMC Task Error: "No slave scheduler found"

BeeGees — Thu, 13 Sep 2018 12:44:55 GMT

I can confirm 100% with Your observations. I did not post a ticket on support, since my workaround works perfectly (and i do not wanna waste my team with calls with support anymore, anyway), but my case is exactly as You described it. Therefore could You please post here in which version of Qlik it will be fixed (casue i assume it will be marked as a bug and fixed in future releases/patches)?

Re: QMC Task Error: "No slave scheduler found"

Daniele_Purrone — Thu, 13 Sep 2018 12:52:40 GMT

Thanks Peter!

Re: QMC Task Error: "No slave scheduler found"

Daniele_Purrone — Thu, 13 Sep 2018 12:54:00 GMT

I will. Just remind me if you don't hear from me or someone else at support in the next couple of weeks, please!

Re: QMC Task Error: "No slave scheduler found"

Daniele_Purrone — Thu, 13 Sep 2018 13:23:35 GMT

Peter, if you haven't done it already, please log a support case for every customer environment. This will help us gain some traction. Thanks!