Replicate Warning message -"Target storage format" is SEQUENCE, but the Control Table 'attrep_cdc_partitions' was originally created in TEXT format

[suraju]

Hi Team, have created a Replicate task with Source endpoint as Oracle & Target Endpoint as Databricks (store changes), have selected the Databricks Target Endpoint file format as "SEQUENCE"

am getting the below warning message while executing the replication tasks, 

The Databricks(Cloud Storage) endpoint's "Target storage format" is SEQUENCE, but the Control Table 'attrep_cdc_partitions' was originally created in TEXT format. Therefore, the file format set in the endpoint will be ignored.

please could you clarify 

1. is the TEXT format defaulted for control tables ?

2. how can we avoid getting this warning message ?

3. will this by any chance impact the other table replicated from source to target ? 

Thanks & Regards

Suresh Raju

[Dana_Baldwin]

Hi @suraju 

Can you please check the attrep_apply_exceptions table on the target to see if there is additional information logged there about this warning?

Thanks,

Dana

[suraju]

Hi @Dana_Baldwin  - I didn't find any data in the exception table. 

Thanks & Regards

Suresh Raju

[Dana_Baldwin]

Hi @suraju ,

I found one other occurrence of this warning reported when using version 2021.5. The solution appears to be to switch to use sequence file format. From the User Guide:

"When using Text as the target storage format, new lines as part of the data values are not supported (due to an Apache Hadoop limitation). Best practice is to use Sequence as the target storage format."

What version of Replicate are you using?

Thanks,

Dana

[suraju]

Hi @Dana_Baldwin - We are already using target file format as 'SEQUENCE'  and I have mentioned the same in my initial post above (second line)

regarding the replicate version, we are using 2022.11.0.475

Thanks & Regards

Suresh Raju

[Dana_Baldwin]

Hi @suraju 

I apologize, I missed that you were already using Sequence as the file format.

Please open a support case so we can look into this for you and involve our internal support team as needed.

Thanks,

Dana

[SwathiPulagam · Accepted solution]

Hi @suraju ,

1. is the TEXT format defaulted for control tables?
A) The default is TEXT format for the attrep_cdc_partitions table

2. how can we avoid getting this warning message?
A) From the 2022.11 version, we display this message when you choose SEQUENCE format in the endpoint level. The reason, we just want to specify even if you choose the SEQUENCE file format it will not be affected by the control tables.

3. will this by any chance impact the other table replicated from source to target? 
A) This message is the informative purpose and has no effect on your task or downstream applications.

Thanks,

Swathi

[suraju]

Thank you @SwathiPulagam for the response.