Last Sunday we encoutered an error situation on a production Qlik server.
The Distribution service went down and would not start anymore continuously reporting error1607.
After some internet searching I found a possible solution that suggested shutting down the services, rename the Distribution service folder under ProgramData\Qliktech and after that restart the services.
We had to set the Qlik services to manual start first and restart the server in order to establish a controlled environment. After the rename the distribution service started, but the same error now popped up on the Management service. We repeated the same steps as we took on the distribution service, stop all services, rename ManagementService folder and restart service. After this rename and restart all services kept running and the management console was accessible again.
All licensing info as well as NAMED and DOC CAL assignments were kept save, but we lost the entire reload schedule that we had constructed on the server (we are not using publisher). It took us about 45 minutes to rebuild it and after that all worked as a treat and kept working.
I was wondering how to find the cause of the service shutdown. We found some info in the event viewer when the service was re-started in the error situation on sunday:
shutting down the service since there are data with invalid encryption. To continue, activate the 'EraseUndecryptableData' config flag by setting it to true in the config file to erase the invalid data.
Found 52 item(s) with invalid encryption. Follows here in the log file:
Exception while decrypting. EncryptedText length=16 chars Exception=System.Security.Cryptography.CryptographicException: Padding is invalid and cannot be removed.
|| at System.Security.Cryptography.CapiSymmetricAlgorithm.DepadBlock(Byte block, Int32 offset, Int32 count) ||
at System.Security.Cryptography.CapiSymmetricAlgorithm.TransformFinalBlock(Byte inputBuffer, Int32 inputOffset, Int32 inputCount) ||
at System.Security.Cryptography.CryptoStream.FlushFinalBlock() || at System.Security.Cryptography.CryptoStream.Dispose(Boolean disposing) ||
at System.IO.Stream.Close() || at SolutionGlobal.Security.CryptoAlgorithm.Decrypt(EncryptedSecret encryptedSecret, Boolean isCurrentAlgorithm)
The other thing I would like to know is how we can prevent the reload schedule to be lost again. We kept the renamed folders for Distribution and Management service so the files are still accessible for inspection and analysis. Any ideas are welkcome and appreciated.
Re: Services dropped. No restart possible, error 1607
I would contact Qlik Support with all that data without hesitation, to see whether it could be a version related issue or otherwise known defect that Qlik is already aware of or working on (or even fixed).