Data Quality: Handling bad data

[Anonymous]

Hi,

Is it possible to handle bad data (e.g. data that does not satisfy datatype functions) in a separate file?

My job reads data from tables passes through tDataMasking, tDataShuffling  and tMap component and updates the data back to the table.

During these transformation in tDataMasking or tDataShuffling component instead of failing, is it possible to handle the exception and write the record to a separate file?

If possible could you pass a reference.

Thanks,

Revathy. 

[Anonymous · Accepted solution]

Hello,

Here is a component tSchemaComplianceCheck which ensures the data quality of any source data against a reference data source. 

Please refer to this component reference:https://help.talend.com/reader/hQS9iH0_runqcivY9yKolQ/bmHjVdNeIm0hgDfQ_aVTXw

Let us know if it is what you are looking for.

Best regards

Sabrina