topic Re: Source capture's performance is low. in Qlik Replicate

Source capture's performance is low.

vinci_wu — Mon, 17 Apr 2023 07:40:12 GMT

On link https://community.qlik.com/t5/Official-Support-Articles/Troubleshooting-Qlik-Replicate-Latency-and-Performance-Issues/ta-p/1929456

The normal performance is

For Oracle Source, with Performance set to TRACE logging, search for the string "read time is". In general for 50 MB, the read speed should be between 15 ms to 30 ms. If it is much larger then you need to investigate why the read to source is slow.

We use Replicate log reader mode and Log Stream Task. But source performance's "read time is" is 497ms for 512000 bytes on Oracle standby source.How to investigate the performance issue. We use Replicate log reader mode and Log Stream Task. Logs showed Source latency is affected sometimes.How to investigate the performance issue?

Below is the log:

"[PERFORMANCE ]T: Completed to read from online Redo log 512000 bytes at offset 00000000012ec400 with rc 1, read time is 497 ms, thread '1' (oradcdc_redo.c:1042)"

Re: Source capture's performance is low.

Steve_Nguyen — Mon, 17 Apr 2023 18:19:25 GMT

@vinci_wu if your reading the 50MB log of 497ms or more then this could indicate that is a slow network

if you check performance does it have read speed of in the 100ms for 50MB?

do you have other process reading from the same redo log that could slow down.

you can check with your DBA to test copy a 50mb log from source to Replicate server to test.

Re: Source capture's performance is low.

vinci_wu — Tue, 18 Apr 2023 07:57:50 GMT

@Steve_Nguyen

Thanks. copy 50MB log---means ssh 50MB log to replicate server?

Actually, we use NFS for "Copy redo logs to temporary folder" and "Replicate has file level access to temporary folder".

Re: Source capture's performance is low.

Heinvandenheuvel — Tue, 18 Apr 2023 22:29:43 GMT

It appears you have slow access to the NFS share from the Replicate server. Is that 497ms an exception or typical? What is the min/max/avg?

Why are you even using "Copy redo logs to temporary folder" ? Was it a case of 'it seemed like a good idea at the time' or was there a compelling reason? (source DB overhead - presumed or measured?)

When you flip back to 'regular' direct / active redolog reading as bfile, what doesthe performance look like?

What is the NFS host? Not AIX I hope (it can have performance issues with 'DIRECT_IO')

hope this helps,

Hein

Re: Source capture's performance is low.

vinci_wu — Wed, 19 Apr 2023 00:57:48 GMT

@Heinvandenheuvel

Thanks for your reply. We use oracle standby for source. At normal situation, replicate tasks read online redo logs and if there is a lot of transaction log at short time, it will copy archive log to NFS, it will be faster than reading online redo log. 497ms is the speed for reading online redo log, it was not for reading archive redo log. NFS can copy 10GB archive log every min. It is faster than reading redo log. NFS is not the performance issue.

Re: Source capture's performance is low.

vinci_wu — Wed, 19 Apr 2023 01:08:08 GMT

Replicate server ping db source reply:

icmp_seq=2 ttl=63 time=0.592 ms

icmp_seq=3 ttl=63 time=0.594 ms

icmp_seq=4 ttl=63 time=0.596 ms

Re: Source capture's performance is low.

vinci_wu — Mon, 24 Apr 2023 14:57:13 GMT

@Heinvandenheuvel

Is that 497ms an exception or typical? What is the min/max/avg?

---497ms is regular speed when reading online redo logs.

source DB overhead - presumed or measured?

---The source latency is 40 min sometimes, and the performance logs showed Source latency is affected.

Re: Source capture's performance is low.

vinci_wu — Mon, 24 Apr 2023 15:09:37 GMT

@Heinvandenheuvel

Performance logs showed :

Source latency 1465.74 seconds, Target latency 1465.74 seconds, Handling latency 0.00 seconds.

So I think source DB overhead.

Re: Source capture's performance is low.

vinci_wu — Tue, 25 Apr 2023 07:12:38 GMT

@Heinvandenheuvel

When you flip back to 'regular' direct / active redolog reading as bfile, what doesthe performance look like?

--- It is PROD. We can't do the test in PROD. By the way, source db is oracle standby, primary and standby both use ASM to store data and archive log. 497ms is speed taht tasks read online redo log in standby ASM, it is not the speed that read the temp file folder which is in NFS.

Could you help?