Unlock a world of possibilities! Login now and discover the exclusive benefits awaiting you.
Oct 23, 2020 2:05:07 AM
Oct 23, 2020 1:56:49 AM
Engine service on Qlik Sense or QlikView server restarts unexpectedly and intermittently, with a logged engine error message like;
Server aborted. Reason for restart: Internal inconsistency, type A
The engine throws a general exception for internal inconsistency type A when it fails to allocate memory from the operating system, as a result of insufficient available memory when processing a Qlik Sense or QlikView app.
REFERENCES
There can be different root causes or contributing factors for a Qlik Sense or QlikView server to end up with insufficient available memory. Reviewing the below areas can give some direction on how to approach resolving or mitigating internal inconsistency Type A errors.
Qlik Sense and QlikView apps remain in memory for a period of time after the last session that used the app has ended, so that they are pre-loaded in memory for returning users. Apps that are infrequently used can end up allocating memory that could be utilized for other apps or calculations. Lowering the app timeout in memory can free a significant amount of memory as apps are unloaded from memory more frequently when not accessed by users.
A lower timeout increases the number of times an app is read from disk to memory, which may result in a longer initial app opening time for users. For small and medium-sized apps it is not expected that app opening time increases significantly when an app is read from disk, while for larger apps it may be noticeable, and keeping apps in memory for a longer time can be beneficial.
App timeout in memory is 8 hours by default. This is a legacy value to compensate for slow read from storage to memory, where reloading app into memory multiple times per day may lead to a perceived slow app opening by business users. On modern server deployments, the read from storage to memory is expected to be fast and loading app into memory more frequently should not lead to performance issues for business users.
App in memory timeout can be configured through Qlik Sense Enterprise on Windows: Engine settings and QlikView: Document Timeout. The timeout can often be successfully adjusted as low as 5-10 minutes. It is recommended to start with a low value and increase in steps until a balanced result is achieved, where memory utilization remains healthy at the same time as the app opening time is acceptable.
In a multi-node deployment, it is possible to apply different configurations for each Engine, which can be utilized in combination with advanced load balancing to allocate different categories of apps to different nodes. This can allow for large apps to remain in memory on dedicated nodes for a longer time while smaller apps can be purged faster.
Inactive user sessions are likely to also have one or more concurrent apps open, which means they can potentially be allocating unutilized memory until reaching idle timeout. By keeping user session timeout short, it can help release memory for apps that are no longer actively consumed by allowing them to reach the app time in memory timeout.
User session timeout can be configured through Qlik Sense Enterprise on Windows: Session inactivity timeout and QlikView: Session Timeout. User session inactivity timeout is 30 minutes by default, which means that a user gets logged out after not making any selections or action for 30 minutes. It is generally not recommended to alter this value, without a review of resource requirements and usage patterns.
If specific use cases require extended user session timeout, it is possible to create different Qlik Sense Enterprise on Windows: Virtual Proxies catering to different use cases through the same Qlik Sense Proxy.
App data models often have fields that are not used in the app visualizations or have high cardinality (Qlik Associative Engine: What is cardinality?), which can add to the app size on file, leading to unnecessary memory allocated when app loaded into memory for user consumption.
For apps published in a production environment, it is recommended to remove fields that are not required in the app and review fields with high cardinality, to optimize the memory footprint of apps and also optimize calculation performance. This may also include excluding records from the data model that are not relevant for the app use case, e.g. historical data that is always excluded in visualization and analysis.
The Qlik Engine allocates memory from the operating system within the boundaries of working set limits, which are set at the start of Qlik Engine service as a percentage of the total memory available on the server. See Technical Brief: The Qlik Associative Engine memory management and CPU usage for more details on how the working set limit applies to Qlik Engine utilization of memory.
Increased user adoption and a growing number of apps leads to larger resource requirements over time, and may eventually require increased memory capacity on servers to manage the workload. Qlik Associative Engine is an in-memory technology that requires physical RAM for high performance, which means that the utilization of operating system virtual memory page files will lead to an immediate degrade in performance due to the latency of page file swapping.
Server sizing requires analysis of the current and expected usage, including estimation of data volumes. This typically requires involvement from a technical architect from either Qlik or a Qlik Partner.