topic Re: Sense App Metadata Analyzer in Archived Groups

App Metadata Analyzer (Windows)

Tyler_Waterfall — Tue, 22 Jul 2025 18:19:11 GMT

We are excited to share the official release version of the Sense "App Metadata Analyzer" app!

This is a "productized" version of the super app that @Daniel_Pilla and @Levi_Turner posted on Qlik Branch (qs-app-metadata-analyzer).

This app iterates over every application metadata endpoint along with several other QRS calls (Nodes, Apps, Proxies, LB audit), ultimately providing a comprehensive dashboard to analyze your application metadata server-wide. This allows you to have a holistic view of the makeup of all of your Qlik applications, enabling you to have awareness at a granular level of the types of applications in your organization. This application is 100% native to Qlik without any installer, and is easy to configure within the Qlik Sense Enterprise environment as the app takes advantage of the existing 'monitor_apps_REST_app' connection to drive all of the REST calls.

As of the Qlik Sense June 2018 release, a new application level metadata endpoint is available. Data is populated for this endpoint per app post-reload in a June 2018+ environment. You can view this application metadata within your own June 2018+ environment at:

http(s)://<server>/api/v1/apps/<GUID>/data/metadata

where <server> is your Qlik Sense Enterprise server and <GUID> is the application ID. Note that the application is not lifted into RAM as metadata is fetched.

Data from this endpoint is derived as part of the app reload process, and therefore does not include any object or expression related metadata. The data from the endpoint includes:

* server metadata including number of server cores, total server RAM

* reload time

* app RAM base footprint

* field metadata including cardinality, tags, total count, RAM size, synthetic keys

* table metadata including fields, rows, key fields, RAM size, circular references

For a deeper analysis of the application including a demo video, please refer to: https://adminplaybook.qlik-poc.com/docs/asset_management/apps/analyze_app_metadata_analyzer.html

June 2020 Update: In version 2.2.1 and forward of the App Metadata Analyzer, a new Alerting sheet has been added, along with two new variables in the load script. The purpose of this sheet and added capability is to make integration with Qlik Alerting as simple as possible. This new capability and view allows for a Qlik administrator to easily see what applications have breached what thresholds (as well as how many) and be quickly alerted on them. It also allows for the administrator to disable alerts for specific applications and mark others as under review, which might have a different cadence. Please see the attached guide for much more detail and example policies/configurations.

Version History

v2.2.2 (July 2020) >
- Replaced the "AppName" field with the "AppNameUnique" field in many visualization objects to properly visualize identically named apps.
v2.2.1 (June 2020) >
- Resolved incremental load issue to include imported apps with metadata that had not yet been reloaded on the server
- Resolved issue with application tag values not appearing
- Resolved autonumber collision issue with "FieldNameUnique" and "TableNameUnique" fields, due to AutoNumber() being used pre and post QVD
- Resolved issue where some threshold breaches were stored to QVD, forcing applications to be reloaded before a change of threshold would take effect
- Removed system fields and system tables
- Modified chart "Field # Records" to "Field Cardinality" on "Dashboard" sheet
- Modified "Threshold Analysis" sheet to use boolean logic for measures over/under thresholds vs a dynamic range
- Added in the sheet "Alerting" for visualizing threshold breaches and allowing for simple tie-ins to Qlik
- Alerting and NPrinting
- Added flagging of circular references
- Added ability to ignore applications from "Alerting" page with specific tag value
- Added threshold reference lines to "Dashboard" objects
v1.1.0 > Fixed Incremental loading bug (field count changes over time) (QB-981)
v1.0.2 > Added back error handling "loop" in load script to handle missing apps (QLIK-98225)
v1.0.1 > Small fix to correct the KPI for Table Footprint
v1.0.0 GA > First official release, compatible with June 2018 Sense and later, target release September 2019
Required Configuration: In the "configuration" section of load script, update the vu_central_node_host_name and vu_virtual_proxy_prefix variables as described in the load script

Sheets:

Dashboard

Threshold Analysis

App Analysis

Alerting

App Availability

Re: Sense App Metadata Analyzer (BETA)

rzenere_avvale — Fri, 14 Jun 2019 16:48:09 GMT

This seems great! I'll give it a try in the next weeks!

Re: Sense App Metadata Analyzer (BETA)

s_kabir_rab — Fri, 14 Jun 2019 16:54:51 GMT

Great stuff. Thank you.

Re: Sense App Metadata Analyzer

rzenere_avvale — Fri, 19 Jul 2019 08:19:41 GMT

Hi @Tyler_Waterfall ,

Is there a way to exclude the empty applications (the ones that have been just created or opened without data)?
In those cases the endpoint returns nothing, and the reloads fails.

I wouldn't like to use a where clause with a fileSize >= (value) since even if empty, the application could have different sizes ~~(i.e. if I upload an attached content)~~

Many thanks,
Riccardo

Re: Sense App Metadata Analyzer

Tyler_Waterfall — Thu, 18 Jul 2019 16:23:14 GMT

Good question @rzenere_avvale . So you want to exclude apps which have no data (no rows) or no size on disk (fileSize) or?

What about apps used to generate QVDs which drop data at the end?

Re: Sense App Metadata Analyzer

rzenere_avvale — Fri, 19 Jul 2019 08:50:05 GMT

Apps that generate QVDs... Depends. Sometimes we leave some statistics fields, so those have rows.
But definitely when the app has no rows/no size on the disk it would be nice to exclude it!

I thought again about the apps with attached files but have no rows, and those cases should already be excluded since the fileSize is the one of the app without attachments.
Unluckily I believe to have an issue with some application that have data and have been reloaded in September 2018, but I need to investigate more in order to understand why.

Taking a look at the response of the http(s)://{server}/api/v1/apps/{GUID}/data/metadata I see there are always (or most of the times?) three $$SysTable. Do you have any idea about what those are?

Riccardo

Re: Sense App Metadata Analyzer

victorbertoldo — Mon, 14 Oct 2019 20:18:40 GMT

Hi Tyler!!! Great stuff man!

But I did't make it.

Can you help me? Thanks.

Victor Bertoldo.

Re: Sense App Metadata Analyzer

Tyler_Waterfall — Tue, 15 Oct 2019 10:49:33 GMT

Try the 1.0.2 version of the App Metadata Analyzer I just published today. @victorbertoldo

Re: Sense App Metadata Analyzer

victorbertoldo — Tue, 15 Oct 2019 11:39:50 GMT

Where is it? I did't find on github.

Re: Sense App Metadata Analyzer

Tyler_Waterfall — Mon, 02 Dec 2019 12:58:16 GMT

@victorbertoldo - Find the app here in the Qlik Monitoring & Management Topic Group under Documents > App Metadata Analyzer

Re: Sense App Metadata Analyzer

Gencho — Fri, 15 May 2020 16:45:09 GMT

Cool, this is really handy thanks so much @Tyler_Waterfall

Re: Sense App Metadata Analyzer

barnabyd — Mon, 18 May 2020 23:05:47 GMT

This looks really useful. I have just downloaded it but it can't find the connector 'monitor_apps_REST_app'. Must be a problem with the Security Rules as the connector is definitely there.

Well done.

Re: Sense App Metadata Analyzer

Huiying — Wed, 17 Jun 2020 06:31:38 GMT

We have been using this app for a while. It is useful.
Recently we added 2 customer-facing nodes and loadbalanced all customers to dedicated nodes.
But after that, we found the metadata in new nodes do not show in App Metadata Analyzer.
These are my investigation steps:
- port 443, normal. QS proxy service listening, nothing else is using the port.
- had quick look on logs of the new nodes, didn't (yet) find something special.
- Tried url http(s)://{server}/api/v1/apps/{GUID}/data/metadata. I can see the metadata of apps in new nodes only in new nodes, not in central node. But this seems not stable. Once I indeed see metadata of those apps in central node, then I quickly reloaded the App Metadata Analyzer. But there is still no data of those apps.

Today I plan to check certificates of new nodes, & configuration changes on the 'old' nodes.

I highly appreciate any hint what else should be checked! @Tyler_Waterfall

Best regards,

Susan

Re: Sense App Metadata Analyzer

rzenere_avvale — Wed, 17 Jun 2020 06:38:55 GMT

Good morning @Huiying ,

personally I had a similar issue in the past, in an environment where I balance apps on specific nodes.
To solve it, I created a new virtual proxy that load balances every node (central + RIMs) and then I linked it to the central proxy. This way (once modified the script to use this virtual proxy) I was able to reload all metadata.
Note that this new virtual proxy is a 'service' one, so no end user is going to use it.

Any other way to solve this kind of issue is very welcome.

I hope this helps,
Riccardo

Re: Sense App Metadata Analyzer

Huiying — Wed, 17 Jun 2020 09:14:54 GMT

Thank you Riccardo!

I tested the method, now url (http(s)://{server}/DedicatedVirtualProxy)api/v1/apps/{GUID}/data/metadata) works on central node, but I still got 'Error: HTTP protocol error 500 (Internal Server Error):' from App Metadata Analyzer's reload script log. And the App Metadata Analyzer has count of apps on the server, but it doesn't have RAM data.

Do you have further suggestions?
Thank you for helping!

Best regards,
Susan

Re: Sense App Metadata Analyzer

rzenere_avvale — Wed, 17 Jun 2020 09:20:32 GMT

Would you mind sharing the complete log? (censored where needed)

Riccardo

Re: Sense App Metadata Analyzer

Huiying — Wed, 17 Jun 2020 09:30:48 GMT

Below is one example. What does 'the app has wrong format' mean?

20200617T112938.472+0300 0553 EXIT FOR // We successfully fetched app metadata and can proceed onward. Woohoo!
20200617T112938.475+0300 0569 NEXT i
20200617T112938.475+0300 0366 LET vAppGUID = Peek('AppID',i,'AppReloadedInJune2018Plus')
20200617T112938.476+0300 0367 For a=1 to 3 // QLIK-98225 reload failure without loop logic.
20200617T112938.476+0300 0368 TRACE Working on App 61ffb289-99e5-4611-84f4-6d1dd6a513aa
20200617T112938.476+0300 0368 Working on App 61ffb289-99e5-4611-84f4-6d1dd6a513aa
20200617T112938.476+0300 0369
20200617T112938.476+0300 0369 LIB CONNECT TO 'monitor_apps_REST_app'
20200617T112939.278+0300 Connected.
20200617T112939.280+0300 0371 RestConnectorMasterTable:
20200617T112939.280+0300 0372 SQL SELECT
20200617T112939.280+0300 0373 "static_byte_size",
20200617T112939.280+0300 0374 "__KEY_root",
20200617T112939.280+0300 0375 (SELECT
20200617T112939.280+0300 0376 "cpu_time_spent_ms",
20200617T112939.280+0300 0377 "__KEY_reload_meta",
20200617T112939.280+0300 0378 "__FK_reload_meta",
20200617T112939.280+0300 0379 (SELECT
20200617T112939.280+0300 0380 "logical_cores",
20200617T112939.280+0300 0381 "total_memory",
20200617T112939.280+0300 0382 "__FK_hardware"
20200617T112939.280+0300 0383 FROM "hardware" FK "__FK_hardware")
20200617T112939.280+0300 0384 FROM "reload_meta" PK "__KEY_reload_meta" FK "__FK_reload_meta"),
20200617T112939.280+0300 0385 (SELECT
20200617T112939.280+0300 0386 "name",
20200617T112939.280+0300 0387 "is_system",
20200617T112939.280+0300 0388 "is_hidden",
20200617T112939.280+0300 0389 "is_semantic",
20200617T112939.281+0300 0390 "distinct_only",
20200617T112939.281+0300 0391 "cardinal",
20200617T112939.281+0300 0392 "total_count",
20200617T112939.281+0300 0393 "is_locked",
20200617T112939.281+0300 0394 "always_one_selected",
20200617T112939.281+0300 0395 "is_numeric",
20200617T112939.281+0300 0396 "comment",
20200617T112939.281+0300 0397 "byte_size",
20200617T112939.281+0300 0398 "__KEY_fields",
20200617T112939.281+0300 0399 "__FK_fields",
20200617T112939.281+0300 0400 (SELECT
20200617T112939.281+0300 0401 "@Value",
20200617T112939.281+0300 0402 "__FK_src_tables"
20200617T112939.281+0300 0403 FROM "src_tables" FK "__FK_src_tables" ArrayValueAlias "@Value"),
20200617T112939.281+0300 0404 (SELECT
20200617T112939.281+0300 0405 "@Value" AS "@Value_u0",
20200617T112939.281+0300 0406 "__FK_tags"
20200617T112939.281+0300 0407 FROM "tags" FK "__FK_tags" ArrayValueAlias "@Value_u0")
20200617T112939.281+0300 0408 FROM "fields" PK "__KEY_fields" FK "__FK_fields"),
20200617T112939.281+0300 0409 (SELECT
20200617T112939.281+0300 0410 "name" AS "name_u0",
20200617T112939.281+0300 0411 "is_system" AS "is_system_u0",
20200617T112939.281+0300 0412 "is_semantic" AS "is_semantic_u0",
20200617T112939.281+0300 0413 "is_loose",
20200617T112939.281+0300 0414 "no_of_rows",
20200617T112939.281+0300 0415 "no_of_fields",
20200617T112939.281+0300 0416 "no_of_key_fields",
20200617T112939.281+0300 0417 "comment" AS "comment_u0",
20200617T112939.281+0300 0418 "byte_size" AS "byte_size_u0",
20200617T112939.281+0300 0419 "__FK_tables"
20200617T112939.281+0300 0420 FROM "tables" FK "__FK_tables")
20200617T112939.281+0300 0421 FROM JSON (wrap on) "root" PK "__KEY_root"
20200617T112939.281+0300 0422 WITH CONNECTION (
20200617T112939.281+0300 0423 URL "https://server/ama/api/v1/apps/61ffb289-99e5-4611-84f4-6d1dd6a513aa/data/metadata",
20200617T112939.281+0300 0424 HTTPHEADER "User-Agent" "Mozilla/5.0 (Windows NT 6.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/39.0.2171.95 Safari/537.36"
20200617T112939.281+0300 0425 )
20200617T112939.732+0300 37 fields found: logical_cores, total_memory, __FK_hardware, cpu_time_spent_ms, __KEY_reload_meta, __FK_reload_meta, @Value, __FK_src_tables, @Value_u0, __FK_tags, name, is_system, is_hidden, is_semantic, distinct_only, cardinal, total_count, is_locked, always_one_selected, is_numeric, comment, byte_size, __KEY_fields, __FK_fields, name_u0, is_system_u0, is_semantic_u0, is_loose, no_of_rows, no_of_fields, no_of_key_fields, comment_u0, byte_size_u0, __FK_tables, static_byte_size, __KEY_root, __extra_,
20200617T112939.736+0300 Error: HTTP protocol error 500 (Internal Server Error):
20200617T112939.736+0300
20200617T112939.736+0300
20200617T112939.737+0300 0427 IF ScriptError<>12 THEN // Successfully found app metadata. If ScriptError = 12, there was an issue fetching metadata
20200617T112939.748+0300 0556 DROP TABLE RestConnectorMasterTable
20200617T112939.748+0300 0557 TRACE Script Error = 12: File 61ffb289-99e5-4611-84f4-6d1dd6a513aa has wrong format
20200617T112939.748+0300 0557 Script Error = 12: File 61ffb289-99e5-4611-84f4-6d1dd6a513aa has wrong format
20200617T112939.748+0300 0558
20200617T112939.749+0300 0558 IF a<3 THEN // QLIK-98225 reload failure without loop logic.
20200617T112939.749+0300 0559 TRACE 'Trying again...'
20200617T112939.749+0300 0559 'Trying again...'
20200617T112939.749+0300 0560
20200617T112939.749+0300 0560 ELSEIF a=3 THEN
20200617T112939.750+0300 0567 END IF
20200617T112939.750+0300 0568 Next a // loop for error handling
20200617T112939.750+0300 0368 TRACE Working on App 61ffb289-99e5-4611-84f4-6d1dd6a513aa
20200617T112939.750+0300 0368 Working on App 61ffb289-99e5-4611-84f4-6d1dd6a513aa
20200617T112939.750+0300 0369
20200617T112939.750+0300 0369 LIB CONNECT TO 'monitor_apps_REST_app'
20200617T112940.410+0300 Connected.
20200617T112940.413+0300 0371 RestConnectorMasterTable:
20200617T112940.413+0300 0372 SQL SELECT
20200617T112940.413+0300 0373 "static_byte_size",
20200617T112940.413+0300 0374 "__KEY_root",
20200617T112940.413+0300 0375 (SELECT
20200617T112940.413+0300 0376 "cpu_time_spent_ms",
20200617T112940.413+0300 0377 "__KEY_reload_meta",
20200617T112940.413+0300 0378 "__FK_reload_meta",
20200617T112940.413+0300 0379 (SELECT
20200617T112940.413+0300 0380 "logical_cores",
20200617T112940.413+0300 0381 "total_memory",
20200617T112940.413+0300 0382 "__FK_hardware"
20200617T112940.413+0300 0383 FROM "hardware" FK "__FK_hardware")
20200617T112940.413+0300 0384 FROM "reload_meta" PK "__KEY_reload_meta" FK "__FK_reload_meta"),
20200617T112940.413+0300 0385 (SELECT
20200617T112940.413+0300 0386 "name",
20200617T112940.413+0300 0387 "is_system",
20200617T112940.413+0300 0388 "is_hidden",
20200617T112940.413+0300 0389 "is_semantic",
20200617T112940.413+0300 0390 "distinct_only",
20200617T112940.413+0300 0391 "cardinal",
20200617T112940.413+0300 0392 "total_count",
20200617T112940.413+0300 0393 "is_locked",
20200617T112940.413+0300 0394 "always_one_selected",
20200617T112940.413+0300 0395 "is_numeric",
20200617T112940.413+0300 0396 "comment",
20200617T112940.413+0300 0397 "byte_size",
20200617T112940.413+0300 0398 "__KEY_fields",
20200617T112940.413+0300 0399 "__FK_fields",
20200617T112940.413+0300 0400 (SELECT
20200617T112940.413+0300 0401 "@Value",
20200617T112940.413+0300 0402 "__FK_src_tables"
20200617T112940.413+0300 0403 FROM "src_tables" FK "__FK_src_tables" ArrayValueAlias "@Value"),
20200617T112940.413+0300 0404 (SELECT
20200617T112940.413+0300 0405 "@Value" AS "@Value_u0",
20200617T112940.413+0300 0406 "__FK_tags"
20200617T112940.413+0300 0407 FROM "tags" FK "__FK_tags" ArrayValueAlias "@Value_u0")
20200617T112940.413+0300 0408 FROM "fields" PK "__KEY_fields" FK "__FK_fields"),
20200617T112940.413+0300 0409 (SELECT
20200617T112940.413+0300 0410 "name" AS "name_u0",
20200617T112940.413+0300 0411 "is_system" AS "is_system_u0",
20200617T112940.413+0300 0412 "is_semantic" AS "is_semantic_u0",
20200617T112940.413+0300 0413 "is_loose",
20200617T112940.413+0300 0414 "no_of_rows",
20200617T112940.413+0300 0415 "no_of_fields",
20200617T112940.413+0300 0416 "no_of_key_fields",
20200617T112940.413+0300 0417 "comment" AS "comment_u0",
20200617T112940.413+0300 0418 "byte_size" AS "byte_size_u0",
20200617T112940.413+0300 0419 "__FK_tables"
20200617T112940.413+0300 0420 FROM "tables" FK "__FK_tables")
20200617T112940.413+0300 0421 FROM JSON (wrap on) "root" PK "__KEY_root"
20200617T112940.413+0300 0422 WITH CONNECTION (
20200617T112940.413+0300 0423 URL "https://server/ama/api/v1/apps/61ffb289-99e5-4611-84f4-6d1dd6a513aa/data/metadata",
20200617T112940.413+0300 0424 HTTPHEADER "User-Agent" "Mozilla/5.0 (Windows NT 6.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/39.0.2171.95 Safari/537.36"
20200617T112940.413+0300 0425 )
20200617T112940.883+0300 37 fields found: logical_cores, total_memory, __FK_hardware, cpu_time_spent_ms, __KEY_reload_meta, __FK_reload_meta, @Value, __FK_src_tables, @Value_u0, __FK_tags, name, is_system, is_hidden, is_semantic, distinct_only, cardinal, total_count, is_locked, always_one_selected, is_numeric, comment, byte_size, __KEY_fields, __FK_fields, name_u0, is_system_u0, is_semantic_u0, is_loose, no_of_rows, no_of_fields, no_of_key_fields, comment_u0, byte_size_u0, __FK_tables, static_byte_size, __KEY_root, __extra_,
20200617T112940.893+0300 Error: HTTP protocol error 500 (Internal Server Error):
20200617T112940.893+0300
20200617T112940.893+0300
20200617T112940.895+0300 0427 IF ScriptError<>12 THEN // Successfully found app metadata. If ScriptError = 12, there was an issue fetching metadata
20200617T112940.905+0300 0556 DROP TABLE RestConnectorMasterTable
20200617T112940.905+0300 0557 TRACE Script Error = 12: File 61ffb289-99e5-4611-84f4-6d1dd6a513aa has wrong format
20200617T112940.905+0300 0557 Script Error = 12: File 61ffb289-99e5-4611-84f4-6d1dd6a513aa has wrong format
20200617T112940.905+0300 0558
20200617T112940.905+0300 0558 IF a<3 THEN // QLIK-98225 reload failure without loop logic.
20200617T112940.906+0300 0559 TRACE 'Trying again...'
20200617T112940.906+0300 0559 'Trying again...'
20200617T112940.906+0300 0560
20200617T112940.906+0300 0560 ELSEIF a=3 THEN
20200617T112940.906+0300 0567 END IF
20200617T112940.907+0300 0568 Next a // loop for error handling
20200617T112940.907+0300 0368 TRACE Working on App 61ffb289-99e5-4611-84f4-6d1dd6a513aa
20200617T112940.907+0300 0368 Working on App 61ffb289-99e5-4611-84f4-6d1dd6a513aa
20200617T112940.907+0300 0369
20200617T112940.907+0300 0369 LIB CONNECT TO 'monitor_apps_REST_app'
20200617T112941.538+0300 Connected.
20200617T112941.541+0300 0371 RestConnectorMasterTable:
20200617T112941.541+0300 0372 SQL SELECT
20200617T112941.541+0300 0373 "static_byte_size",
20200617T112941.541+0300 0374 "__KEY_root",
20200617T112941.541+0300 0375 (SELECT
20200617T112941.541+0300 0376 "cpu_time_spent_ms",
20200617T112941.541+0300 0377 "__KEY_reload_meta",
20200617T112941.541+0300 0378 "__FK_reload_meta",
20200617T112941.541+0300 0379 (SELECT
20200617T112941.541+0300 0380 "logical_cores",
20200617T112941.541+0300 0381 "total_memory",
20200617T112941.541+0300 0382 "__FK_hardware"
20200617T112941.541+0300 0383 FROM "hardware" FK "__FK_hardware")
20200617T112941.541+0300 0384 FROM "reload_meta" PK "__KEY_reload_meta" FK "__FK_reload_meta"),
20200617T112941.541+0300 0385 (SELECT
20200617T112941.541+0300 0386 "name",
20200617T112941.541+0300 0387 "is_system",
20200617T112941.541+0300 0388 "is_hidden",
20200617T112941.541+0300 0389 "is_semantic",
20200617T112941.541+0300 0390 "distinct_only",
20200617T112941.541+0300 0391 "cardinal",
20200617T112941.541+0300 0392 "total_count",
20200617T112941.541+0300 0393 "is_locked",
20200617T112941.541+0300 0394 "always_one_selected",
20200617T112941.541+0300 0395 "is_numeric",
20200617T112941.541+0300 0396 "comment",
20200617T112941.541+0300 0397 "byte_size",
20200617T112941.541+0300 0398 "__KEY_fields",
20200617T112941.541+0300 0399 "__FK_fields",
20200617T112941.541+0300 0400 (SELECT
20200617T112941.541+0300 0401 "@Value",
20200617T112941.541+0300 0402 "__FK_src_tables"
20200617T112941.541+0300 0403 FROM "src_tables" FK "__FK_src_tables" ArrayValueAlias "@Value"),
20200617T112941.541+0300 0404 (SELECT
20200617T112941.541+0300 0405 "@Value" AS "@Value_u0",
20200617T112941.541+0300 0406 "__FK_tags"
20200617T112941.541+0300 0407 FROM "tags" FK "__FK_tags" ArrayValueAlias "@Value_u0")
20200617T112941.541+0300 0408 FROM "fields" PK "__KEY_fields" FK "__FK_fields"),
20200617T112941.541+0300 0409 (SELECT
20200617T112941.541+0300 0410 "name" AS "name_u0",
20200617T112941.541+0300 0411 "is_system" AS "is_system_u0",
20200617T112941.541+0300 0412 "is_semantic" AS "is_semantic_u0",
20200617T112941.541+0300 0413 "is_loose",
20200617T112941.541+0300 0414 "no_of_rows",
20200617T112941.541+0300 0415 "no_of_fields",
20200617T112941.541+0300 0416 "no_of_key_fields",
20200617T112941.541+0300 0417 "comment" AS "comment_u0",
20200617T112941.541+0300 0418 "byte_size" AS "byte_size_u0",
20200617T112941.541+0300 0419 "__FK_tables"
20200617T112941.541+0300 0420 FROM "tables" FK "__FK_tables")
20200617T112941.541+0300 0421 FROM JSON (wrap on) "root" PK "__KEY_root"
20200617T112941.541+0300 0422 WITH CONNECTION (
20200617T112941.541+0300 0423 URL "https://server/ama/api/v1/apps/61ffb289-99e5-4611-84f4-6d1dd6a513aa/data/metadata",
20200617T112941.541+0300 0424 HTTPHEADER "User-Agent" "Mozilla/5.0 (Windows NT 6.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/39.0.2171.95 Safari/537.36"
20200617T112941.541+0300 0425 )
20200617T112941.988+0300 37 fields found: logical_cores, total_memory, __FK_hardware, cpu_time_spent_ms, __KEY_reload_meta, __FK_reload_meta, @Value, __FK_src_tables, @Value_u0, __FK_tags, name, is_system, is_hidden, is_semantic, distinct_only, cardinal, total_count, is_locked, always_one_selected, is_numeric, comment, byte_size, __KEY_fields, __FK_fields, name_u0, is_system_u0, is_semantic_u0, is_loose, no_of_rows, no_of_fields, no_of_key_fields, comment_u0, byte_size_u0, __FK_tables, static_byte_size, __KEY_root, __extra_,
20200617T112941.994+0300 Error: HTTP protocol error 500 (Internal Server Error):
20200617T112941.994+0300
20200617T112941.994+0300
20200617T112941.995+0300 0427 IF ScriptError<>12 THEN // Successfully found app metadata. If ScriptError = 12, there was an issue fetching metadata
20200617T112942.009+0300 0556 DROP TABLE RestConnectorMasterTable
20200617T112942.010+0300 0557 TRACE Script Error = 12: File 61ffb289-99e5-4611-84f4-6d1dd6a513aa has wrong format
20200617T112942.010+0300 0557 Script Error = 12: File 61ffb289-99e5-4611-84f4-6d1dd6a513aa has wrong format
20200617T112942.010+0300 0558
20200617T112942.010+0300 0558 IF a<3 THEN // QLIK-98225 reload failure without loop logic.
20200617T112942.011+0300 0561 AppEndpointsNotFound:
20200617T112942.011+0300 0562 LOAD
20200617T112942.011+0300 0563 '61ffb289-99e5-4611-84f4-6d1dd6a513aa' AS AppID,
20200617T112942.011+0300 0564 1 AS AppEndpointNotFound
20200617T112942.011+0300 0565 AUTOGENERATE(1)
20200617T112942.011+0300 2 fields found: AppID, AppEndpointNotFound,
20200617T112942.012+0300 1 lines fetched
20200617T112942.012+0300 0566 END IF
20200617T112942.012+0300 0567 END IF
20200617T112942.013+0300 0568 Next a // loop for error handling

Re: Sense App Metadata Analyzer

Huiying — Thu, 18 Jun 2020 10:08:57 GMT

@Levi_Turner
Could you give a hint where to check for the 'HTTP protocol error 500' message in App Metadata Analyzer reload script log?

We use Qlik Sense Enterprise 2020 Feb version on a 8-node cluster. We check App Metadata Analyzer every now and then.
2 weeks ago, we added 2 customer-facing node, and loadbalanced all customers to own dedicated customer-facing node.
This week we found that for the 2 new nodes & their reload node, App Metadata Analyzer has app count but doesn't have RAM data.
URL (http(s)://{central node}/{dedicated virtual proxy}/api/v1/apps/{GUID}/data/metadata) works fine from browser.

The whole cluster works normally. New nodes also work normally from both hub & mashup.

My investigation steps:
- port 443: normal, no blocker, listening by QS proxy service
- certificates: normal
- created dedicated virtual proxy for App Metadata Analyzer. The virtual proxy loadbalances all RIM nodes. After reload, error is still in the script log.
- log: quickly went through logs on the new nodes, but didn't find yet something I should focus on now.

I feel out of ideas where to check next. Highly appreciate any help!

Best regards,
Susan

Re: Sense App Metadata Analyzer

rzenere_avvale — Thu, 18 Jun 2020 10:49:45 GMT

Good morning @Huiying ,

just to be sure:

- created dedicated virtual proxy for App Metadata Analyzer. The virtual proxy loadbalances all RIM nodes. After reload, error is still in the script log.

Is the new virtual proxy load balancing all RIM nodes AND central node?
If not, for what I've seen in the past, it's possible to have different results from browser and script call (also the response can change from time to time)

Riccardo

Re: Sense App Metadata Analyzer

Huiying — Thu, 18 Jun 2020 10:55:31 GMT

@rzenere_avvale yes, the dedicated virtual proxy load balances all 8 nodes in the cluster, including central node, proxy node, customer-facing nodes & reload nodes.
When checking the url in browser, I agree that it is not stable. I sometimes get it connected, sometimes not.
But for script, it has never worked I think, even when url works