topic Monitoring Load/CDC Statistics - Extract or Logs in Qlik Replicate

Monitoring Load/CDC Statistics - Extract or Logs

simonB2020 — Tue, 16 Aug 2022 17:22:02 GMT

Regarding the statistics (Insert/Update/Delete/DDL) for each table in all of Tasks, full load & CDC.
I have a need to get these details out to an external system.

So what are my options ?

I see that there is an API (api_getTableStatus) that I could use to poll at intervals, though I would also have to first list tasks and then iterate.

Do these stats get written to a log file anywhere ?
Any other options available?

Thanks for any insights ....

Re: Monitoring Load/CDC Statistics - Extract or Logs

Dana_Baldwin — Tue, 16 Aug 2022 17:28:34 GMT

Hi @simonB2020 ,

Have you considered our Enterprise Manager product? It will collect these stats and write them to a Postgres database, more details can be found here:

https://help.qlik.com/en-US/enterprise-manager/May2022/Content/EnterpriseManager/Main/Analytics/Analytics.htm

Thanks,

Dana

Re: Monitoring Load/CDC Statistics - Extract or Logs

simonB2020 — Fri, 26 Aug 2022 18:01:20 GMT

Dana,

Unless I've missed something, this only appears to give granularity to down to the task level, nothing for individual tables.

Do you know how I can the stats for tables please ?

Thanks

Re: Monitoring Load/CDC Statistics - Extract or Logs

lyka — Fri, 26 Aug 2022 19:05:25 GMT

Hi @simonB2020

You can use the GetTableStatuses API and you can write a script around it and direct the output to a text file. But yes, you have to do it for each task. I have seen this being used in the field wherein customers with write a script to get the table status per task.

More information here:

https://help.qlik.com/en-US/enterprise-manager/May2022/Content/EnterpriseManager/EnterpriseManager_APIGuide/CurlAPI/api_getTableStatus.htm

Retrieves the tables statuses of a specific Replicate task for all tables that match the specified state(s), table schema(s), and table name(s). This is useful for automation processes, for example, as it allows you to retrieve tables in a certain state (for example, suspended) and then perform an operation on them (for example, ReloadTable).

There is also a control table called Replication status but this will also give task details and not individual tables but still have really useful information.

Also, if you download a diagnostic package, you will see a file called tables_status.json

This will have the table status information at the time of the download of the DP.

Hope this helps!

Thanks

Lyka