topic Re: tlogcatcher not catching any error in talend cloud in Talend Studio

tlogcatcher not catching any error in talend cloud

NNayal1600240775 — Fri, 15 Nov 2024 21:16:09 GMT

Hi,

I have an error 'Value too large for a column...."

I have not selected 'Die on Error' in tOracleOutput Component.

When I used to run this Job in Talend Enterprise , the tlogcatcher used to catch this error and send out an email

Now we have migrated the Job and this flow doesn't even run in case of this kind of error. I do not want to select die on error in toracleoutput, as we want the job to continue irrespective of any kind of rejections.

tlogcatcher is not catching these errors in talend cloud

Re: tlogcatcher not catching any error in talend cloud

Anonymous — Wed, 08 Nov 2023 07:59:24 GMT

Hi @Neha Nayal

If 'die on error' option is unchecked, the Java exception is handled internally and it isn't thrown out, so tLogCatcher can't catch them. You can monitor the rejects flow and send email if it has data.

Regards

Shong

Re: tlogcatcher not catching any error in talend cloud

NNayal1600240775 — Wed, 08 Nov 2023 08:00:51 GMT

If I incorporate the reject flow the job that loades 30000 records within 5 minutes takes longer than 2 hours with reject link. Hence i have not used to reject link.

Re: tlogcatcher not catching any error in talend cloud

Anonymous — Wed, 08 Nov 2023 08:22:48 GMT

You are right, the reject link can't be used with batch insert, the performance drops significantly for large data volumes.

Re: tlogcatcher not catching any error in talend cloud

NNayal1600240775 — Fri, 17 Nov 2023 07:33:07 GMT

Hi @Shicong Hong ,

when i was using the Talend Enterprise edition, i didnt select die on error option, and when there was any column which was rejecting because of datalength issue those exceptions the tlogcatcher used to catch those exception and send an email and process all the other records.

Whereas in talend cloud the same job , is not catching any exception in tlogcatcher.

Is there any solution to this how i can achieve it without selecting die on error and reject row

Re: tlogcatcher not catching any error in talend cloud

Anonymous — Fri, 24 Nov 2023 02:51:29 GMT

"when i was using the Talend Enterprise edition, i didnt select die on error option, and when there was any column which was rejecting because of datalength issue those exceptions the tlogcatcher used to catch those exception and send an email and process all the other records."

@Neha Nayal , sorry for the delaying reply. You meant tLogCatcher capture the Java exception error even though you uncheck the 'die on error' option of tOracleOutput when the job is execution in Studio?

Re: tlogcatcher not catching any error in talend cloud

NNayal1600240775 — Fri, 24 Nov 2023 05:12:59 GMT

Hi @Shicong Hong ,

The tlogCatcher should capture this error value too large for column irrespective of die on error unselected. but the log catcher fails to do so in talend cloud edition.

Re: tlogcatcher not catching any error in talend cloud

Anonymous — Fri, 24 Nov 2023 08:04:04 GMT

@Neha Nayal

As I told you in previous post, If 'die on error' option is unchecked, the Java exception is handled internally and it isn't thrown out, so tLogCatcher can't catch them, this is not a component bug, it is a jog design issue.

If you want to capture the error and let the job continue to execute next record, you need to re-design the job as below:

main job:

tFileInputDelimted(data source)--main--tFlowToiterate--iterate--tRunJob

tRunJob: call the child job, pass the current row to child job (please refer to this page to learn how to pass a value from main job to child job), uncheck 'die on error' option on tRunJob.

child job:

tFixedFlowInput(generate the current row)--main--tOracleOutput_1

tLogCatcher--main-...>tOracleOutput_2(error table)--runIf--tSendMail.

on tOracleOutput_1, check 'die on error' option.

Note: for large data volumes, the performance of this design is relatively low.