QMC affected by numbers of fiels in EDXResult

[waszcma1]

Hi,

I have a problem with QlikView May 2021 SR2

There is something wrong with synchronization of the services and xml files generation in EDXResult folder

Week ago I discovered that non of the task can be triggered either manualy or by scheduler they seems to be frozen when I pressed the run it took couple second to proccess the action and back to prevoious stage without any logs.

After checking forum I have found that somone is recommending clear the folder C:\ProgramData\QlikTech\DistributionService\EDXResult from older files but system was constantly frozen when I tryied to entry to that folder, IT said that there must be incredible high number of files even more then 200k or 1 mln,

so we delete the whole EDXResult folder and create empty one with the same name -it works QMC start working as normal

but checking this folder I see every day increase in number of files over 26000 a day ! it is not acceptable as after couple of days we have more than 100 000 fiels and the same problems comming.

What do you recommend to do as a first action? what could be a root cause of generation so many files every day?

p.s.
After when I delete the folder and run only one small task which normaly takes 35 sek it generate 100 + fiels in after when task was completed.

and most of this files looks like this:

<?xml version="1.0" encoding="UTF-8"?>
<Root FinishTime="" StartTime="" TaskLogFilePath="" TaskStatus="Waiting" TaskId="0a254874-5bf7-4f56-a0cc-e613ab3cee51" ExecId="00019809-6230-4d83-86fa-15063cf31d43"/>

[marcus_sommer]

Like in the above thread hinted it's most likely that any status-messages from tasks of being waiting/continuing/queued or similar stuff are recorded. For me it seems that such behaviour (such huge number of files within keep-days-window) is rather seldom and therefore probably caused from any customized settings. So going through the own documentation of configuring the environment could give the appropriate hints.

Beside this you could monitor the behaviour more closely to find the time/application/tasks which are causing most of the issue, for example you load these (xml) edx-files into QlikView with extra information like the filetime(). In further steps you may combine this with the event-, performance- and application-logging of QlikView and the OS.

[NadiaB]

Hi @waszcma1 , @KHSDM , here is some documentation about the log creation:

EDXResult: This folder holds the EDX result files created for every task. The Help says: Until the task is completed, this file holds the status of the EDX task. When the execution is finished, it holds the result (success/fail) and the task started as a result (if any). These are created for all tasks so the number of files can increase exponentially, and it may be wise to decrease the following .exe.config setting to a much smaller number than the default 30 days.

The document from where the information was extracted is the following:

https://support.qlik.com/servlet/fileField?retURL=%2Fapex%2FQS_CaseWizardKnowledgeArticle%3FarticleI...

Hope it helps!

[Skage]

@NadiaB 

I facing the same issue.

After an upgrade from an old version of QlikView to May 2023 SR1 I'm facing the same in a customer-environment.
Before the upgrade only an handful of logs were created but now the system i spewing out over 100k per day. Apx 60 per minute.

Prior to the upgrade one full year would create apx 100k files so the increase is 365 times.

Reducing the NbrOfDaysToKeepEDXResultFiles to a day or so might "improve" the situation but the key question is why does 60 new files get created per minute without any valuable information?

Is there anything one can do to stop this from happening?
Why doesn't the log say anything about what the file actually handle?

Is there a way to determine what name the task with the provided taskid really is?

[marcus_sommer]

By combining all log-files (not just the EDXResults) from the distribution-engine you could map the task-id's with the relevant task-names and also which applications are behind them. Further you would get much more information what's going on and probably also some hints why there are now so many files generated which may also led to any related configurations. 60 new files per minute could mean that the current (execution/queued/waiting)-state is stored per each second and not only during the real run-time of any task.

As a starting-point you may use the QlikView 12 Application: System Monitor - Qlik Community - 1497281. At least the previous releases of this tool did contain various load-statements to the distribution log-files and should simplify such task significantly.

[ekech_infomotio]

Are there any news on this problem? One of my customers is also affected and getting some hundred thousand EDXResult files per day.

In comparison, another customer with a similar sized environment and even more active+scheduled tasks, but a quite old/outdated release has only about 8000 EDXResult files over 30 days.

For now we mitigated the problem by changing the config for keeping the EDXResults from 30 to 1,  but it's not a real solution... 

[ekech_infomotio]

After my customer did some task and trigger cleanup (reviewing and re-applying the majority of the task configuration), the QDS now seems to be working "normally" again and NOT producing hundreds of thousands of useless files.

It's just an assumption, but for me it looks like one or more task configurations got corrupted (in a weird and not visible way) which led QV to this strange behaviour.

So, to resolve the problem: check your tasks, check your triggers and hit apply even if you didn't change anything at all.