Replicate Task for Oracle to S3 and Oracle to HANA is crashing abnormally

[akaradhya]

Hi Team, 

In the recent time, I have been running into this task abnormally crashing more frequently. 

Background: We are replicating data from Oracle to S3 and Oracle to HANA (This is for a ecommerce giant). However, there is this single table(Orders) which sits in a separate task that is crashing abnormally and since the task is crashing, no logs are generated on the Replicate side. However, we tried to collect the Linux core dump for which the setup is already in place. For some reason, when the task is crashing, linux too is not collecting any logs. 

This happens only from Friday night through Sunday evening which is when there are a lot of logs generated at the source because of the high user access. 

The current resolution we have is to manually extract the last processed SCN and run the task from the next SCN number. Surprisingly, during the weekdays when we try to start the task from where it failed on a Friday or Saturday night...it seems to work fine. 

However, this manual process is only a temporary fix, and we are looking for a permanent solution. Support ticket has been raised and the support has asked for Linux core dump which we unfortunately were not able to generate. 

Requesting the community to throw some light on this to help us with the root cause. 

Thanks, 

Krishna A. 

[Steve_Nguyen]

@akaradhya 

what is the version of Replicate your using , exact version ?

what is the version of Oracle ?

what is the Oracle Endpoint setting ?

[akaradhya]

Hi Steve, 

Please find the requested information

Replicate version: 2022.11.0.571

Oracle Version: Oracle 19.15

Oracle Endpoint: We have added an internal parameter readAheadBlocks and set it to 500000. 

[Steve_Nguyen]

@akaradhya there is a known issue with 2022.11 kit 

https://community.qlik.com/t5/Official-Support-Articles/Qlik-Replicate-with-Oracle-End-point-Resume-...

[akaradhya]

@Steve_Nguyen  We were previously on 2022.5.0.815 version and we were facing the same issue. My understanding from the investigation I did with the logs is that, the task works fine when it is reading the archive logs. The moment the task switches to read from the online logs is when we are facing this issue. Again, this does not happen with all the online log reads but only with a few of them. 

[Steve_Nguyen]

@akaradhya 

are you sure you are on 2022.11.0.571 ?

because current release is  Replicate 2022.11 SP04 (2022.11.0.546)

This could be that you are on a non GA release .  we should have the 2022.11 SP05 release with week, you should upgrade to be on GA release first.

[akaradhya]

Yes @Steve_Nguyen we are on 2022.11.0.571. We were aware that this is a non GA release. We received this information from the support team to upgrade to this version in order to handle another data related issue with HANA and that is the reason we have upgraded to this version. 

[Steve_Nguyen]

@akaradhya since you are on a non GA kit, open a support case , you may need to set source_capture ,,, verbose logging ,, and set /trace verbose to memory and set for 200MB,, if issue happen .

1. open case to support

2. zip and attach task log to case and task Diagnostic Package.

[akaradhya]

Sure @Steve_Nguyen  Appreciate your responses.