topic Re: repctl checkpoint "task name" for downstream task (source: log stream task & target : oracle database) is not giving correct timest in Qlik Replicate

repctl checkpoint "task name" for downstream task (source: log stream task & target : oracle database) is not giving correct timestamp

dipankar — Wed, 08 May 2024 08:23:23 GMT

repctl checkpoint "task name" for downstream task (source: log stream task & target : oracle database) is not giving correct timestamp.

D:\bin>repctl getcheckpoint "TASK_INGCDC_02"

command getcheckpoint response:

{

"checkpoint": "checkpoint:V1#33512#1271;638506460899807820;20240120221751514712#0#0#*#0#137"

}

[getcheckpoint command] Succeeded

Regards

Dipankar

Re: repctl checkpoint "task name" for downstream task (source: log stream task & target : oracle database) is not giving correct timest

sureshkumar — Wed, 08 May 2024 09:06:12 GMT

Hello @dipankar

Kindly test it with

repctl -d "data location” getcheckpoint "task_name"

Check whether you are getting the same issue or not.

Regards,

Suresh

Re: repctl checkpoint "task name" for downstream task (source: log stream task & target : oracle database) is not giving correct timest

john_wang — Wed, 08 May 2024 09:11:26 GMT

Hello Dipankar @dipankar ,

Thanks for reaching out to Qlik Community!

I guess I did not get the question very well. Would you please elaborate the exact doubt?

Regards,

John.

Re: repctl checkpoint "task name" for downstream task (source: log stream task & target : oracle database) is not giving correct timest

SushilKumar — Wed, 08 May 2024 09:55:54 GMT

Hello @dipankar

First that getting result from Repctl is not Recommend for customer usage. It is Advised in certain cases wherever its required. There are lot of timestamps which update along with the data replication.

Regards,

Sushil Kumar

Re: repctl checkpoint "task name" for downstream task (source: log stream task & target : oracle database) is not giving correct timest

dipankar — Wed, 08 May 2024 10:11:25 GMT

Environment Information :

A log stream (upstream) staging task is created which is reading from source endpoint ( oracle ) and storing changes onto the replicate server
There are around 20 downstream tasks whose source endpoint is log stream staging task and target endpoint is oracle.
Qlik console provides on latency information which doesn’t helps us much as per requirement of our environment.
We Need precise timestamp information to know uptill which point in time data has been committed in the target database.
We used repctl checkpoint “task name” to get the information about the checkpoint.

Problem statement

Detailed output of repctl command for downstream tasks shared below

D:\bin>repctl getcheckpoint "TASK_INGCDC_02"

command getcheckpoint response:

{

"checkpoint": "checkpoint:V1#33512#1271;638506460899807820;20240120221751514712#0#0#*#0#137"

}

[getcheckpoint command] Succeeded

D:\bin>

D:\bin>repctl getcheckpoint "TASK_INGCDC_03"

command getcheckpoint response:

{

"checkpoint": "checkpoint:V1#33483#1271;638506460899807820;20240120221751514712#0#0#*#0#137"

}

[getcheckpoint command] Succeeded

We reached out to engineers from qlik professional services (Michael Chiu) to help us decode output of above repctl command. They reviewed this output and told us that above output translates to below time stamp

The timestamp is "202401202217515147", which is "2024-Jan-20 22:17:51.514"

These downstream tasks have been created couple of days ago and went online (started replicating) yesterday (07^th May ’24 ) so the checkpoint/commit provided for these downstream tasks is too old.

Below is response from Michael Chiu to this problem

“I have just double-checked, turns out the "20240120221751514712" is not the timestamp of the last change written to the target, it is the name (based on the creation time) of the sub-directory under the "..\LOG_STREAM\audit_service" parent directory,

that is why it is unchanging.”

Expected solution

We want to able to retrieve timestamp information uptill which downstream tasks have committed/written data to the target database.
This commit time stamp information is important as we have to take some decision while generating business critical reports.

Re: repctl checkpoint "task name" for downstream task (source: log stream task & target : oracle database) is not giving correct timest

SachinB — Wed, 08 May 2024 12:22:02 GMT

Hello @dipankar ,

Thanks for reaching out to Qlik Community!

If you are looking for last timestamp updated record in change table at target endpoint, then you can refer the "[header__] timestamp" for store change table, which give you the clarity when the last record has been updated.

The above example seems to be fetching "Timeline" folder names from Log stream task, this will generate when you resume the task with timestamp.

Hope my understanding is correct here, Else we can discuss on this further through the case.

Regards,

Sachin B

Re: repctl checkpoint "task name" for downstream task (source: log stream task & target : oracle database) is not giving correct timest

Heinvandenheuvel — Wed, 08 May 2024 14:44:44 GMT

note: This comment is just there to prevent other helpers from going down an incorrect path.

At first I thought you might need 'connect; ' on you command line to get live data vs static from the SQLite repositories like : data\tasks\<yourtask>\task_table.sqlite --> cdc_status. Checking with gettaskstatus you may well see 'STOPPED" while you know the task is running. checkpoint status might be the exception because in fact you need it when there is no more live server to connect to.

The Replicate doc shows

repctl -d "C:\Program Files\Attunity\Replicate\data” getcheckpoint "dg1task"
repctl -d connect; execute ....

No connect on the first command. And indeed, when you try this first command against a running system it works with or without connect. returning the same values. But when you try against a stopped system it works cleanly without connect but with connect, that connect command (of course!) fails. The getcheckpoint still works.

Now for the actual problem. When you execute the getcheckpoint against the original source tasks which feeds the logstream is that result as expected?

Hein.

C:\scripts>repctl -d \Replicate\data connect; getcheckpoint oracle_to_oracle [connect; command] Succeeded Failed to connect to localhost:3550: The operation completed successfully. (0) (exit status 1003518) command getcheckpoint response: { "checkpoint": "checkpoint:V1#229#00000000.0a9af1a1.00000001.0000.00.0000:876.222804.16#0#0#*#0#0" } [getcheckpoint command] Succeeded