QlikSense on-prem hybrid deployment service stuck in crash loop

[qliktime]

In a customer environment without internet access we have a few times (lets say 3-6 months apart) observed that suddenly 90+% of CPU and RAM is consumed. It is not by the Qlik services but by Windows internal services, such as "System" or "WmiProviderService".

We've resolved it by rebooting the server, but we have noted it because the QlikSense environement becomes completeley unusable.

I might have found the culprit. The hybrid deployment service on this server normally outputs a log file that says "Features are not supported by the license", which is expected since this customer has no need for hybrid.

This generated about 1 mb log files per day.

Now I noted that the daily log file suddenly goes up to 100 mb and it is filled with an endless loop of dotnet crash failing to do a postgres schema migration.

My suspicion is that postgres bombards the windows event log and somehow overloads some internal services in Windows after a while. I'm not familiar enough with Windows OS to know if this is reasonable or not, but it is my best hypothesis.

Have anyone observed anything similar?

How to resolve?

269389 20241115T163415.095+01:00 ERROR SERVERNAME 82 DOMAIN\serviceaccount Connection id ""0HN7HLRKMQA2R"", Request id ""0HN7HLRKMQA2R:00000002"": An unhandled exception was thrown by the application. Marten.Exceptions.MartenCommandException: Marten Command Failure:$↵↓DO $$
BEGIN
IF NOT EXISTS(
SELECT schema_name
FROM information_schema.schemata
WHERE schema_name = 'hybrid_deployment_service'
)
THEN
EXECUTE 'CREATE SCHEMA hybrid_deployment_service';
END IF;

[rzenere_avvale]

Hey there @qliktime ,
In your scenario, where Hybrid deployment is not required, you could also disable the microservice. See the following kb article: How to disable the App Distribution Service (ADS) on a machine in a multinode environment.
Once done, you'll observe in your Event Viewer a warning message, that says that the microservice is disabled.
Just keep in mind that at each release upgrade, you'll need to check if those settings have been kept or not.

I hope this helps,
Riccardo