topic Full Load Failed in Qlik Replicate

Full Load Failed

gseckin — Thu, 04 Jul 2024 07:54:32 GMT

Hi,

I created a task with cdc + full load with 11 tables. 10 tables loaded but one is failed during full load. I also enabled parallel load, but it didn't help. I'm using oracle as a source and kafka as a target. When I try to find logs related to this failure, I can not see anything. This table contains more than 4b rows. Do you have a configuration for big tables. How can I load big tables without error?

And where can I find the logs?

My last question is that can I start this full load from the point it stopped?

Thanks

Re: Full Load Failed

john_wang — Thu, 04 Jul 2024 07:59:39 GMT

Hello @gseckin ,

Thanks for reaching out to Qlik Community!

The detailed error messages recorded in Task log files, depends on Qlik Replicate platform, the log files can be found in below location

Windows - C:\Program Files\Attunity\Replicate\data\logs

Linux - /opt/attunity/replicate/data/logs

BTW, if an error occurs during the Full Load stage then we need to redo the Full Load from scratch.

Hope this helps.

John.

Re: Full Load Failed

gseckin — Thu, 04 Jul 2024 08:04:55 GMT

Hi @john_wang ,

Yes I know the log path, and in the path you said. I also found log files. But I can not find something related the full load failure. It says Task is running, and actually cdc part is still running. I need to find full load log. Do you have different log file for full load?

And my sec question is do you have a configurations to load big tables?

Thanks

Re: Full Load Failed

john_wang — Thu, 04 Jul 2024 08:11:51 GMT

Hello @gseckin ,
Thanks for the update.

You may change the logging level of different logging components. In order to trace the Full Load stage issue, you may set SOUCE_UNLOAD/TARGET_LOAD to Trace, or Verbose to get detailed information, see below screen sample:

Save the changes, re-run the task and then check the task log files.

Regards,

John.

Re: Full Load Failed

SushilKumar — Thu, 04 Jul 2024 08:13:21 GMT

Hello @gseckin

IF the Full load task is running then its highly likely QR is fetching the Data to process the data further.

There are Some values you can change to extract the data faster, Parallel work when table have partition or try to get the data divide by the parallel config in full load settings.

Regards,

Sushil Kumar

Re: Full Load Failed

gseckin — Thu, 04 Jul 2024 08:16:23 GMT

I found this configuration to set log level but it's different than yours

Where I can find the one you showed me?

Re: Full Load Failed

john_wang — Thu, 04 Jul 2024 08:17:24 GMT

And my sec question is do you have a configurations to load big tables?

I'm not very sure what's the Qlik Replicate version you are running now, I'd like suggest you to use Parallel Load to speed up the Full Load.

Regards,

John.

Re: Full Load Failed

gseckin — Thu, 04 Jul 2024 08:18:53 GMT

Hi @SushilKumar ,

Yes already did that configuration. Increased max table count to 30 and enable pararlel load for table. Let me share my configurations.

This is task settings

and this is table settings

What am I missing here? Can you check my configurations? Thanks

Re: Full Load Failed

vinayak_m — Thu, 04 Jul 2024 08:31:11 GMT

Hi @gseckin ,

Partition option will work when the partitions are already created in the source system.
Since you've already tried this and it does not seem to work, I suggest you try to use data ranges on a datetime column preferably.
And segment the data of your table on the specific datetime column.(Year/Date)
You can also use any ID column or PK column if present in table.

Please see image for reference,
Let us know if this configuration works for you.

Re: Full Load Failed

gseckin — Thu, 04 Jul 2024 10:16:27 GMT

I just saw this error,

ORA-01555 snapshot too old: rollback segment number 565 with name :..... too small. How we can fix this issue? Thanks

Re: Full Load Failed

john_wang — Thu, 04 Jul 2024 10:25:15 GMT

Hello @gseckin ,

There are several articles for ORA-01555:

ORA-01555: Snapshot too old: rollback segment number

Qlik Replicate: ORA-01555: snapshot too old

Hope this helps.

John.

Re: Full Load Failed

vinayak_m — Thu, 04 Jul 2024 10:27:35 GMT

Hi @gseckin ,

This is an oracle error, you have to get in touch with the Oracle DBA or the table owner and see if any change is happening.

This happens when the data is written/modified in the table while the table is being extracted, try running the task again at a later time or off business hours.