Full reload and CDC failed with message. Failed to get source column def for ids[2]=1335435064

[desmondchew]

We have an existing old task that has failed under "error". We did a full reload it works. But after several minutes, the CDC kicks in and we got the error message as below. I have replaced the schema and table name. The rest are the error logs.

Table 'Schema'.'table123' (subtask 0 thread 1) is suspended. Failed to get source column def for ids[0]=6.

The source endpoint is Oracle 11g and target endpoint is MySQL 8 AWS. We are on version 

• November 2022 (2022.11.0.628)

Update:

I am getting this error now.

 Failed to get source column def for ids[2]=1335435064

[SachinB]

Hello @desmondchew ,

Thanks for reaching out Qlik community !!

If reload is not working, and again it's failing with CDC process, We would request you to remove the table, Save the task and again readd to the task. Do a normal resume.

If issue still persist, We would request you to set source capture and Target apply in verbose, Attach table DDLs from source and Target databases and create support case for further investigation.

Thanks for understanding !!

Regards,

Sachin B

[desmondchew]

I have done the export table to JSON file. Import into a new task and reload. This table has 250 columns but we are taking partial columns. So importing using JSON file is the quickest. 

I have done 5-6 times reload and it has failed when it goes into CDC. 

Desmond

[SachinB]

Hello @desmondchew ,

Exporting and reimporting same JSON will carry same dynamic metadata file, So I would request you to remove and readd the table to the task and See how it goes.

Regards,

Sachin B

[desmondchew]

This is an existing task that has been running for years. There was no change to the task, no ddl, filter, transformation, or changes. So that's why I am puzzled why we are taking this approach of redoing. The table has 925 columns, out of that we need 70 columns. 

Anyway, I have stopped the task and creating a new one with the required columns. I am doing a reload and see if the error goes away.

Desmond

[desmondchew]

Ok, the new task with a new table added manually, and 75 columns were manually added. Reloaded and having the same error.

[Heinvandenheuvel]

Something must have changed right? Replicate version change?

Since the task ran for years, and a fresh definition didn't fix anything, I have to think something changed on the source. If I were a gambling man, I'd put my money on more columns having been added.

Anyway, can we see a bit more of the full context of the error, preferably  10 or 20 log lines  before the error and a few after with full details - timestamps, facility (SOURCE_CAPTURE? ), source module/line (tabledef.c)? 

Please present in 'code' box or as .TXT attachment, not screenprint, and fee free to obscure names if that feels desirable.

Hein