QEM service can't start - Failed to connect to SQLite repository 'rtdata.sqlite'

[NakulanR]

Hi Support,

We have recently seen an issue where the Qlik Enterprise Manager service fails to start with the QEM logs indicating the following error:
"[ERROR] REPO-E-FACSQLREP, Failed to connect to SQLite repository 'C:\Program Files\Attunity\Enterprise Manager\data\rtdata.sqlite' "

We were able to resolve the issue by stopping the QEM service and renaming the rtdata.sqlite file to rtdata_old.sqlite, and then renaming the rtdata.new.sqlite file to rtdata.sqlite. After then running the command to upgrade the repository (aemctl.exe repository upgrade), the QEM service was able to run and the console could be accessed.

Previously when we have seen an issue with a data repository file it has been a result of memory or disk space issues on the server. However, in this case, there have been no reported memory/disk space issues on the server. What we'd like to understand is what are the possible reasons the rtdata.sqlite file becomes unreadable/corrupted.

This instance of Enterprise Manager is v2023.11.0.259

Thanks,

Nak

[Dana_Baldwin]

Hi @NakulanR 

It is difficult to say.

Has the disk been checked for errors?

Was there any change or issue in the environment around this time?

Is any software such as antivirus scanning any of the Qlik application or data files?

Dana

[SushilKumar · Accepted solution]

Hello @NakulanR 

Such Error occurs ""[ERROR] REPO-E-FACSQLREP, Failed to connect to SQLite repository 'C:\Program Files\Attunity\Enterprise Manager\data\rtdata.sqlite' ""

due one process is reading the SQLite file and its terminated abnormally.  So you would need to check which session or process holding that file. as all the SQLite file hold internal info for QEM .

Regards,

Sushil Kumar

[NakulanR]

Hi @Dana_Baldwin & @SushilKumar ,

We haven't seen any errors or warnings on the server/disk, and there are no known anti-virus scanners that scan through the Enterprise Manager files. The only change in the environment was that Windows patching was due to occur slightly before the issue arose, but the patching process was halted due to an unknown issue. A little while after the patching process was stopped, the Enterprise Manager service stopped working.

Could a Windows process be holding that file as a result of the Windows patch? If so, would the process be holding on to all the QEM repository files rather than just the rtdata file?

Regards,

Nak

Regards,

Nak

[Dana_Baldwin · Accepted solution]

Hi @NakulanR Difficult to say, but something else had a lock on the file. The only way to know is to check what held the lock at the time it happened. Process Monitor from Microsoft SysInternals could help here: File and Disk Utilities - Sysinternals | Microsoft Learn

Thanks,

Dana

[NakulanR]

Hi @Dana_Baldwin ,

Thanks for the suggestion, we'll look into that. We haven't seen the issue since, but will definitely try the sysinternals utility should the issue occur again.

Regards,

Nak

[SushilKumar]

Hello @NakulanR 

As per previous experience whenever Windows patching happens on the QEM or Replicate we Advise to stop services and start the services. because Window patches mostly developed in C++ and  .NET Framework ,  and modify the way they interact the resources (CPU/memory/Process)

It is obvious when patch process executes it  may pertain to such behaviour. you can check the same with Event viewer around that time. and read the Patch info for more info as which Window component may be touched or altered by patch .

Regards,

Sushil Kumar

[SachinB]

Hello @NakulanR ,

Make sure there aren't any software applications scanning the QEM directories.

This may lead to lock the respective files.

Regards,

Sachin B