Rejecting invalid data records and write bad records to a file
The need is to import CSV files and load the data into database and write all bad/invalid data to a separate csv file. My question is what is the best way to do this? Based on my understanding, I can either use "Trigger-> On Component Error" or use tMap. Just wonder what is a better approach between these two or is there any other approach. Thanks.
How are you determining the validity or your data? Talend offers several components you could use.
For example there is a tSchemaComplianceCheck component.
"Validates all input rows against a reference schema or checks type, nullability, length of rows against reference values. The validation can be carried out in full or partly."
