Skip to main content Resources | Qlik Greenway
cancel
Turn on suggestions
Auto-suggest helps you quickly narrow down your search results by suggesting possible matches as you type.
Showing results for 
Search instead for 
Did you mean: 

Qlik Community

Ask a Question

Knowledge

Search or browse our knowledge base to find answers to your questions ranging from account questions to troubleshooting error messages. The content is curated and updated by our global Support team

Announcements
Now Live: Qlik Sense SaaS Simplified Authoring – Analytics Creation for Everyone: READ DETAILS

Operations Monitor or License Monitor Tasks Do Not Reload - Updated and Verbose - Monitoring Apps

cancel
Turn on suggestions
Auto-suggest helps you quickly narrow down your search results by suggesting possible matches as you type.
Showing results for 
Search instead for 
Did you mean: 
Andre_Sostizzo
Digital Support
Digital Support
‎2016-09-01 08:19 AM

Operations Monitor or License Monitor Tasks Do Not Reload - Updated and Verbose - Monitoring Apps


An additional Article with other helpful information: How to troubleshoot and resolve reload and task issues with monitoring apps

 

Common Errors and links to the related fixes:


There are several reasons for failed Operations Monitor or License Monitor reloads in Qlik Sense 3.x and 11.x (2016/2017). Please see below summary for most common symptoms and references to articles with more details. 

  • Make sure that Data Connections connects to QRS.
    There are 2 possible scenarios:
    • HTTPS port is not the default 443, has been customized to a different number in the Proxy.
    • HTTP is used by ticking Allow HTTP in the Proxy.
      Typical error messages: "Error: QVX_UNEXPECTED_END_OF_DATA: Failed to connect to server" or QVX_UNEXPECTED_END_OF_DATA: The remote server returned an error: (404) Not Found. 
  • Central node virtual proxy has been configured with "Form authentication" and monitor app reload log shows an error message like "QVX_UNEXPECTED_END_OF_DATA: Unexpected character encountered while parsing value: <. Path '', line 0, position 0?.?"

 

  • If you receive "Qlik Sense Operations Monitor A problem fetching data from QRS via the REST connector"
This article also helps with failed monitor app reload on RIM node in a multi-node environment, check QVX_UNEXPECTED_END_OF_DATA: Failed to connect to server


Additional Resources:

 

Resolution:

 

To fix the monitoring app issues in any version of Sense is to make the changes and updates below:

  1. First, it's very important to verify that the Service Account is added to your proper AD Group (check with your Active Directory team) and is an admin, especially if your User Directory Connector in the QMC uses an LDAP filter.

  2. Temporarily disable antivirus or provide an exception for all Qlik folders

  3. Open all ports in the firewall that are needed according to the only documentation for the June 2020 release.

  4. Check that all services are running and can be restarted, as well as verifying the user account running them. If any service is not running, reboot the server. The account running the services must be the dedicated Qlik service account created for this Qlik installation--verify it has not been changed to any other user such as a company employee.

  5. There are 3 main directories for Qlik Sense, the one in "C:\Program Files\Qlik", the one in “C:\ProgramData\Qlik\Sense”, as well as the shared folder (if using Shared Persistence). Right-click on these two directories and verify that the Qlik Service account used to run the Qlik services is listed on the Security tab and has "Full Control". In addition, check the "Sharing" tab as well to make sure this same account is listed under file sharing and has "Read/Write" abilities.

  6. Finally, verify that the Qlik Server has internet connectivity. It must be able to reach the outside world. This is a requirement for versions June 2017 – February 2018. If there is no internet access via a browser to the outside world, then all monitoring apps will fail. Please see  An error occurred / Failed to load connection error message in Qlik Sense - Server Has No Internet in the support portal if you have no internet access.

  7. Now that the pre-req's have been met, it's time to dive into the admin portal by opening the QMC portal

  8. Delete all connections in the "Data Connections" tab that begin with "QRS_" (before June 2017) AND "Monitor_Apps_" (after June 2017). If the Sense version is June 2017 or later and for some reason the "QRS" and the "Monitor_Apps" data connections exist, then delete all of them.

  9. Next, go to the Apps tab and delete all monitoring apps that are named License Monitor, Operations Monitor, etc. (related to monitoring).

  10. Minimize the QMC. This step is very important and required: Delete all ".QVD" files within the Windows Directory that's related to the monitoring apps (usually C:\ProgramData\Qlik\Sense\Log"). Do not open any directories, merely look at the files in this folder. All files related to "reloads", "governance", "monitor", "etc... any QVD file listed in this Log folder needs to be deleted except ones that have the terms "central logging" in them.

  11. In addition, go to the shared ArchivedLog shared folder in Windows and ZIP all log files especially the Engine logs, as anything older than 6 months can sometimes cause the reloads to fail. Once a ZIP is done, delete all log files. This is usually not required but is helpful in cases where the log files are too big and are causing reload issues.

  12. Now that the monitoring apps have been cleaned up, as well as the underlying files causing issues, open the QMC again.

  13. In the Apps section of the QMC, import the new default monitoring apps using the "Import" button within the "Apps" section of the QMC. C:\ProgramData\Qlik\Sense\Repository\DefaultApps

  14. Next, check the Virtual Proxy to see if they use a prefix. If so, this prefix needs to be included in the connection string for each “monitoring_app” Data Connection. (e.g. https://qlikserver1/prefix123/qps instead of https://qlikserver1/qps ; assuming the Virtual Proxy prefix is named “prefix123”.

  15. Check the Proxy and verify the HTTPS Default port is 443. If not, update the DC connection strings.

  16. Check the Proxy and verify that Kerberos Authentication is not enabled.

  17. It is required that at least one virtual proxy uses the Windows authentication pattern for the monitoring apps to work. Just confirm you are using "Windows" in the Virtual Proxy settings under the "Authentication" section.

  18. After which, click on "Tasks" under "Associated Items" for each Monitoring App that was imported and create an automatic rule to have them automatically refresh. Set the time for the first start to 1 minute from now so that the trigger will automatically kick off once you save. After running through all of these steps, be sure to delete this rule afterwards so that they don't try to load every 1 minute.

  19. Go to the "Data Connections" section of the QMC to verify that new "QRS_" (before June 2017) or "Monitor_Apps_" (June 2017+) data connections were automatically re-created.

  20. Open one of them and view the entire connection string. Verify that this exists in the string ";skipServerCertificateValidation=true;". Now verify all of the other "monitor_apps_" data connections to check that this is listed in the connection string. This is often the cause of issues post-upgrade.

  21. If it’s a multi-node setup and newer than Sense 3.2.5, then the ServerLogFolder and ArchivedLogFolder must also be updated per this link.

  22. Verify if https://localhost (June or newer: the URL's are actually listed twice in each data connection and both must be updated; in v 3.2.5 or older it's listed only once) or the FQDN should be listed in each connection string and change if needed. FQDN is required for multi-nodes or cloud solutions – or when files are not local. It's highly suggested to never use localhost. Review the beginning of the connection string for each "monitor_app" data connection as well as the end, since the URL (e.g. localhost or FQDN) will be listed twice and both need updating.

  23. Open each Data Connection beginning with "QRS_" (before June 2017) or "Monitor_Apps_" (after June 2017), and update the User ID and Password fields (must use the Service Account running the Qlik Services… look in Windows Services). Note: this service account must be an administrator with Full Control and be a part of the required AD Groups listed in the installation guide. Example – if the Qlik Services show the user running these services is “domain123\qvservice” then enter “domain123\qvservice” as the User ID in each “monitor_app” Data Connection, and then enter the password for this service account. Repeat for each “monitor_app” data connection. This is also one of the main causes of monitoring app issues, especially if you get a "4xx" error such as a "401 Unauthorized". A "401 Unauthorized" is generally caused by the wrong User ID, wrong Password, or leaving the User ID as default (.\admin).

  24. On a related note, if the Qlik Services within Windows Services show the user account is local, such as “.\qlikservice”, then the User ID that will need to be entered in the QMC for every Data Connection that begins with “monitor_apps_” will need to be (servername\serviceuser). For example, if the central server’s name is “Server123”, and the Qlik Services show that “.\qlikservice” is the service account running Qlik Services, then enter in “Server123\qlikservice” as the User ID in the QMC for each data connection.

  25. It's time to check if everything is now working properly. Go to the Apps section, click on the monitoring apps that were imported, choose "More Actions" and click "Reload Now".

  26. Go to the "Tasks" tab to see if they reloaded. Hit refresh in the top right of the QMC page.

  27. If the version of Sense is "November 2017" or "11.24.x", there is a code related issue that effects some installations. If it’s an upgraded November 2017 release and the above didn’t fix the issue (be sure to apply the latest November Patch before continuing), try using a copy of the September 2017 monitoring app QVF files. You can get these from Qlik Support. In the QMC go ahead and import them, then replace the November monitoring app files. The Data Connections and QVD files (as mentioned above in detail!) will need to be deleted first, before attempting to load the September release of the monitoring apps.

  28. This issue should now be addressed. Reload the tasks manually and collect the Script Reload Logs for review.

  29. If the reloads are now working, be sure to edit the scheduled reload times for the monitoring apps. As noted above, we set this to occur every minute for testing purposes. It's suggested to change this to once daily or weekly.

  30. In case of the reload tasks still failing, download the script reload logs on this page for the tasks that fail, and add it to the support case for further review.

 
 


Related Helpful Articles:

 
 
Other major items to check, verify, or change:

  • Tasks Ensure that the connection string for each monitoring app data connection has https://localhost listed instead of “http” (it lists this twice in each long connection string in June 2017 and later), unless you have a multi-node environment in which you MUST use the FQDN of the central server instead of “localhost”.
  • Check if the service accounts are interactive. Both the user that logged into the system and the service account needs access to the data connectors for the monitoring tools.
  • If you did an upgrade and are using the June version or newer (11.11.x or higher) of Qlik, the names of the connections in the Data Connections tab of the QMC should now begin with “monitor_apps_” and not “qrs_”; however if you’re running Qlik 3.2.5 or older than they should be listed as “qrs_”. If you are running the June version or newer and do not see “monitor_apps_” Data Connections listed, then that would be the issue.
  • The “Owner” of these monitoring apps’ Data Connections should be listed as “sa_repository(Internal\sa_repository), which is the default owner.
  • Verify that the “ArchivedLogsFolder” item (with the Data Connection tab of the QMC) is present.
  • Verify that this exists in the DC mon apps connection strings ";skipServerCertificateValidation=true;"
  • Have them verify that the Service Account is added to their proper AD Group and is an admin. Related to this, go to UDC and check the LDAP filter. If the service account is not a part of the group listed for their filter (e.g. if service account is not in the group “CN=QlikUsers”, then the service account will not have rights to the monitoring apps. A temp fix is to remove the filter, ensure “existing users” box is checked, load, and ensure service account is pulled in. the long term resolution is to ensure service account is added to the AD Group listed in filter.
  • If you are using a “Prefix” within the Virtual Proxy section of the QMC, this prefix must be appended to the data connection. This is usually the cause of issues if you upgrade your Qlik version and use a Prefix for your virtual proxy. For example, if you use a Virtual Proxy Prefix called “prod” then your data connection would resemble “url=https://localhost/prod/qrs/app” instead of “url=https://localhost/qrs/app”.
  • Check the Proxy and verify the HTTPS Default port is 443. If not, update the DC connection strings.
  • The service account must have administrator privileges and be added in “local users and groups” to the following groups: Qlik Sense Service Users and Performance Monitor Users
  • The service account must be assigned Full Control permissions for the folder %ProgramData%\Qlik\Sense
  • If the service user account is changed, you must run the proper bootstrap commands and also follow THIS article related to Changing the user account type to run the Qlik Sense services on an existing site
  • The service account must exist in the QMC as a User and also be set as a RootAdmin
  • If you go to edit one of the data connections such as "monitor_apps_rest_user", verify that the UserID and Password is correct. It should be the default account used to install Qlik, or the admin service account that are running the Qlik Services if you go to the "Services" area of the server and look at the user running the Qlik services. This is often the case of monitoring apps still failing if the rest of the steps do not work. Re-type in the User ID and Password for each UDC and save, to see if things now work, as in client environments they change passwords often but do not update the QMC as necessary. All “monitor_apps_” data connections would need the password updated. This user (in each Data Connections) listed as the User ID must also be set as a “RootAdmin” (when editing the user in the “Users” section of the QMC).
  • Delete all monitoring apps and qrs/monitor_apps data connections then go to "C:\ProgramData\Qlik\Sense\Log" and you will see files such as"Operations_Monitor_Reload_Stats_3.1.0.qvd". Zip up all of these files into one ZIP and then delete them all. This may resolve issues related to corrupt .qvd file.
  • The Rest connector file "QvRestConnector.exe.config" file may be missing from its default location. This rest connector error can be tested and replicated by double clicking on the "QvRestConnector.exe" file directly within the installation folder "C:\Program Files\Common Files\Qlik\Custom Data\QvRestConnector". An example URL to use is http://maps.googleapis.com/maps/api/geocode/xml?address=1600+Amphitheatre+Parkway,+Mountain+View,+CA... then click OK to see if you get data on the next screen.
  • In QMC Virtual Proxies / Authentication, change he "Windows authentication pattern" to "Form" instead of "Windows" then login to the Hub as the Service user to replicate the service user running the app. Change it back to “Windows” afterwards instead of “Forms” though.
  • For the Operations Monitor and License Monitor to reload correctly, it is required that at least one virtual proxy uses the Windows authentication pattern.
  • If the version of Qlik is June or newer, monitoring apps will fail if “Kerberos Authentication” is enabled within the QMC-->Proxy section.
  • In the November 2017 release, there is a code related issue. If it’s an upgraded November 2017 (non-patch) release then use a copy of the September 2017 mon app QVF files, in the QMC import them, then replace the November mon app files. The Data Connections and QVD files in the folder may or may not need to be deleted; trial and error.
  • If it’s a multi-node setup and newer than Sense 3.2.5, then the ServerLogFolder and ArchivedLogFolder must also be updated per this link.
Labels (1)
Labels:
24,096 Views
Comments
dwighttrumbower
Creator
Creator
‎2021-02-10 02:03 PM

user_map:
20210210T123825.171-0600 0701 Mapping LOAD
20210210T123825.171-0600 0702 [__FK_user]&'user' as key,
20210210T123825.171-0600 0703 LOWER(userDirectory & '\' & userId) as UserId
20210210T123825.172-0600 0705 SQL SELECT
20210210T123825.172-0600 0706 (SELECT
20210210T123825.172-0600 0707 "id",
20210210T123825.172-0600 0708 "userId",
20210210T123825.172-0600 0709 "userDirectory",
20210210T123825.172-0600 0710 "__FK_user"
20210210T123825.172-0600 0711 FROM "user" FK "__FK_user")
20210210T123825.172-0600 0712 FROM JSON (wrap on) "root" PK "__KEY_root"
20210210T123826.149-0600 2 fields found: key, UserId,
20210210T123826.171-0600
20210210T123826.171-0600 Unexpected character encountered while parsing value: <. Path '', line 0, position 0.
20210210T123826.171-0600 Error: Unexpected character encountered while parsing value: <. Path '', line 0, position 0.
20210210T123826.171-0600 Unexpected character encountered while parsing value: <. Path '', line 0, position 0.
20210210T123826.172-0600 Execution Failed
20210210T123826.200-0600 Execution finished.

Version history
Last update:
‎2021-02-10 07:06 AM
Updated by:
Digital Support
Contributors