Hello Folks,
I am very new to talend development. I have very good experience in SSIS (SQL Server ETL tool). In SSIS, while processing the records, if any particular component fails due to an error in data like data conversion error or truncation of data or any other error, SSIS has the flexibility to redirect the rows that failed to process into a file or table by defining the on error component in the package.Is it possible to the similar workflow in Talend? I have looked at tconverttype which only handles the errors related to data conversion. Is there any component or solution to design a generic solution to redirect the rows failed to process to a table or file using Talend? I really appreciate all your inputs.
Thanks in advance!!!
Hi,
Most components in Talend support the Reject row (make sure you disable the Die on Error option).
Usually I prefer to have a tMap component which receives the Reject row and maps some columns to an output (most of the time including the complete input row). This output is then sent to a Subjob that does generic error logging, like saving all data to a database table.
Hope this helps.
Regards,
Arno
Thanks Arno for your quick reply. I have already enabled the reject row setting on the tmap but its not working. I have added a screenshot related to it. I am not sure whether i have configured it correctly or not.How to create a subjob that gets executed when the error occurs?
Thanks!!
Hi, > How to create a subjob that gets executed when the error occurs? well, as you've said, you can create a subjob that executes in case of an error (I don't know if it will solve your problem) component -> another component | | onError \/ error subjob
I have the same issue, only the setup is a bit different... I have a tFileInputExcel that has a schema, when someone accidentally puts the wrong file, an error will show up ...
Exception in component tFileInputExcel_1 org.apache.poi.openxml4j.exceptions.InvalidOperationException: Can't open the specified file:
I want to catch this error, remove it from the console and dump it to a log file, so in the console everything will look like a normal run. tLogCatcher does the dumping but how do i suppress the error from the console?
I want to catch this error, remove it from the console and dump it to a log file, so in the console everything will look like a normal run. tLogCatcher does the dumping but how do i suppress the error from the console?
You need to move this processing into a child job.
In child job, check the option 'die on error' on tFileInputExcel and use tLogCatcher to dump it to a log file.
In parent job, use a tRunJob to call the child job and uncheck the option 'die on error' on tRunJob. You will not see the error message on the console when running the parent job.
Best regards
Shong
nope... I still get the error in the console when running the parent...
connecting to socket on port 3609 connected Exception in component tFileInputExcel_1 org.apache.poi.openxml4j.exceptions.InvalidOperationException: Can't open the specified file:
nope... I still get the error in the console when running the parent...
connecting to socket on port 3609 connected Exception in component tFileInputExcel_1 org.apache.poi.openxml4j.exceptions.InvalidOperationException: Can't open the specified file:
Hi
Yes, you are right. The exception messages are still printed on the console even though it is called from a parent job.
Maybe the tSystem component could fix you need, export the job script and use a tSystem component to execute the job script in another job. On tSystem, select 'normal' in both Standard Output and Error Output list.
