Qliksense scheduler: queued tasks should not be assigned to a scheduler node
Hello
When you submit a Qliksense task and all schedulers reach their "max concurrent reloads" limit, the task is put into a queue of a specific scheduler node. In my opinion the task should be put into a generic queue and the scheduler nodes, when "available", should get the tasks to be executed from that generic queue. In this way we should get a better workload distribution, better performance and.. availability: for my understanding at the moment if a scheduler node dies, all queued tasks linked to that node will not be executed.
I'm adding another use case we are facing right now. Two of our 4 schedulers crashed. Tasks were pilling up in the queue for hours. Once we fixed the issue and started the 2 other schedulers, they were taking in charge the new tasks but not the one queued as they are already allocated to a node. It means that if we were trying to have a smart way of managing our ressources and start additional servers when the queue starts to grow, this would not work.
NOTE: Upon clicking this link 2 tabs may open - please feel free to close the one with a login page. If you only see 1 tab with the login page, please try clicking this link first: Authenticate me! then try the link above again. Ensure pop-up blocker is off.
