Rejecting a file

[Anonymous]

i have a job in which input is a comma separated values file. I need to check is all the data is valid, for example. length , data type , compare to schema etc. I tried t schema compliance check component but it separates the rejected rows from accepts ones.

But I need to reject the entire file if at least even one of validation fails i.e ( even one of the row fails ). The rejected files must be saved to another location . Please help

[TRF · Accepted solution]

Hi,

So you need to realize all the validations to verify your file using the component you want such astSchemaComplianceCheck, tFilterRow or other. For each component, use the reject flow to push the rejected records somewhere (tFileOutputDelimited or tHasOutput for example).

At the end, get the number of rejected records and if it's > 0 you can push the entire input file where you want.

If you want to stop the job after the 1rst rejected record, use a tWarn or tDie on the reject flow.

Add a tLogCatcher to catch this event, push the input file where you want and stop the job.