topic abort job when duplicates are found in Talend Studio

abort job when duplicates are found

Anonymous — Tue, 23 Oct 2018 14:16:33 GMT

Hi, I have a situation when I have to abort the job when the duplicates are found and not load it to output file.

I have below logic:

In case duplicates are found, the job dies. but in case there are no duplicates, the job doesnt load the output file. I pass a dummy variable on tfixedflowinput(which i think i m doing wrong), as it doesnt update the file with 4 fields from source. how to do it..? Thanks!

Re: abort job when duplicates are found

Anonymous — Tue, 23 Oct 2018 15:36:53 GMT

it is better to make use of the after variables of the tUniqRow which tells the number of uniques and number of duplicates found from input , you can keep the uniques in temporary and update only if the number of duplicates are zero, and yes tFixFlowInput would not work that way

Re: abort job when duplicates are found

Anonymous — Tue, 23 Oct 2018 15:51:31 GMT

something like this would work i guess

Re: abort job when duplicates are found

Anonymous — Tue, 23 Oct 2018 16:14:06 GMT

@CK395, can u give export of this job?

when i do IF, it doesnt give me option of tdie. is there anything special u did..? Neither do i get option for thashinput & thashoutput

Re: abort job when duplicates are found

Anonymous — Tue, 23 Oct 2018 17:31:07 GMT

Hi,

Here is an alternate option to resolve your problem. Add a context variable "row_count" in integer format.

Job details with taggregate component details

loading count to a context variable

The tjava component connected to On Subjob Ok is dummy. The if component connected to tDie is :-

context.row_count !=0

and the if condition connected to the next flow will be having the condition

context.row_count ==0

Warm Regards,

Nikhil Thampi

Re: abort job when duplicates are found

Anonymous — Tue, 23 Oct 2018 20:52:11 GMT

@nthampi - Thanks for the logic. what did u put in tfixedflow input..?

Re: abort job when duplicates are found

Anonymous — Wed, 24 Oct 2018 05:46:38 GMT

Hi,

I didn't notice that earlier skeleton diagram was missing the data capture of unique records.

You can store the unique records in a format of your choice (either as file or hash) and read them later for downstream processing.

If the answer has helped to answer your query, please mark the topic as resolved. Kudos are also welcome 🙂

Warm Regards,

Nikhil Thampi

Re: abort job when duplicates are found

Anonymous — Wed, 24 Oct 2018 08:18:51 GMT

my bad i didn't see the choose files button earlier,

Anyways, the job is attached, make sure to change the file paths and configure components according to your schema.

let me know if it fulfilled your requirement or not.

regards

Chandra Kant

DieIFDuplicate.zip