topic Re: Qlik Replicate: PostgreSQL as source | Heartbeat update info in Qlik Replicate

Qlik Replicate: PostgreSQL as source | Heartbeat update info

dineshkumarl — Tue, 30 Apr 2024 12:59:02 GMT

Hi All,

I am replicating data from PostgreSQL to AWS S3 and have set heartbeat to 5 mins (Default).

At source (PostgreSQL) I see the heartbeat table column "hb_last_heartbeat_at" is still 1 PM IST.

I have 2 tables in the task currently. Kindly help me understand how the column timestamp update.

Will the heartbeat only update if there are no changes in both the tables (or more) or will it update even when there is no change in any of the table within 5mins.

Looking for detailed explanation, thanks

Re: Qlik Replicate: PostgreSQL as source | Heartbeat update info

john_wang — Tue, 30 Apr 2024 13:34:06 GMT

Hello @dineshkumarl ,

Thanks for reaching out to Qlik Community!

Not sure if the task is in running status? In general if the heartbeat is enabled, as long as the task is running, it will write the "keep alive" periodical events in despite of other table has changed data or not.

Regards,

John.

Re: Qlik Replicate: PostgreSQL as source | Heartbeat update info

john_wang — Tue, 30 Apr 2024 13:56:04 GMT

Hello @dineshkumarl ,

Some information from my labs:

1. In the PostgreSQL source endpoint, set the heartbeat interval to 1 minute:

it's seen in task log file:

2024-04-30T21:38:35:628273 [SOURCE_CAPTURE ]I: Heartbeat mechanism is set to 1 minutes (= 60 seconds) at schema 'public'

2. Query the heartbeat table we can see the value is updated in every minute:

Hope this helps.

John.

Re: Qlik Replicate: PostgreSQL as source | Heartbeat update info

dineshkumarl — Wed, 01 May 2024 07:10:32 GMT

Hi @john_wang Thanks for the information. The task is in running state and I have kept the heartbeat to default 5 mins. However, the heartbeat doesn't update in 5 mins, it takes longer than 5 mins. Can be 6min or 10 mins or 40 mins or 2 hours, not sure on why it's taking longer.

I Qlik replicate task logs I can see the heartbeat set interval and when I query the table the result isn't expected.

I will test it out by setting the heartbeat to 1mins and let you know my findings.

Thanks.

Re: Qlik Replicate: PostgreSQL as source | Heartbeat update info

john_wang — Wed, 01 May 2024 14:02:28 GMT

Hello @dineshkumarl ,

Thanks for the update.

I've conducted a sanity test with Heartbeat frequency set to 5 minutes in Qlik Replicate 2023.11.0.282, it works for me, see below twice sending Keep-Alive entries in task log file (I removed some lines between the UPDATE operations):

You might set SOURCE_CAPTURE to Verbose and keep the task running for about 15 minutes then check the task log file to see if any clue.

Please note that my PostgreSQL database is hosted on-premise rather than in the cloud. If you require assistance, please open a support ticket and attach the Diagnostic Packages. Our support team will be more than happy to assist you (Please share the case number here, I will look into it as well).

Thanks,

John.

Re: Qlik Replicate: PostgreSQL as source | Heartbeat update info

DesmondWOO — Thu, 02 May 2024 07:15:45 GMT

Hi @dineshkumarl ,

As John mentioned above, when Replicate updates the attrep_heartbeat table greater than 5 minutes, please enable verbose logging on SOURECE_CAPTURE. By compare the timestamp printed by Replicate in the task log with the actual the timestamp of the attrep_heartbeat table, we can identify whether the delay is caused by the Replicate itself or by the database.

Regards,
Desmond

Re: Qlik Replicate: PostgreSQL as source | Heartbeat update info

dineshkumarl — Wed, 08 May 2024 11:55:08 GMT

I did test it out. It might be a temporary issue which we faced. After setting the heartbeat to 1min and gradually increasing the time frame, its working as expected. Thank you for your assistance.

Re: Qlik Replicate: PostgreSQL as source | Heartbeat update info

john_wang — Thu, 09 May 2024 03:01:03 GMT

Thank you so much for the feedback @dineshkumarl ! Glad to hear it works for you .