topic Re: [resolved] Wait for Multiple Component Oks in Talend Studio

[resolved] Wait for Multiple Component Oks

Anonymous — Sat, 16 Nov 2024 12:26:27 GMT

I am working on a job where I create a file, process it, and then I want to delete it. I want to handle the potential for reprocessing a file, so I perform a lookup to see if the record exists and then insert/update accordingly. I am iterating over a list of files that are created. So if there are 5 files, I want to process each one individually and after it is done processing, I want to delete it. The problem that I am having is that the delete step is executed after the first ComponentOk hits it and not the last. I tested it with using the "Insert or Update" and "Update or Insert", but the performance was not as good. It took almost twice as long to complete. I thought about moving the delete until after the subjob finishes, but in the case of an error, the files that were successful would not be removed. I appreciate any thoughts on this.

Re: [resolved] Wait for Multiple Component Oks

Anonymous — Thu, 19 Jan 2012 13:07:26 GMT

Hi,
You could first insert or update the datas and then do a subjob to delete the files which are in the folder.
tfileList-----your job without the deletion
|
on subjob ok
|
tfileList----tfileDelete

Or you could also put your job in an other job. So your job will look like this
tRunJob_1---tFileList----tRunJob_2----oncomponentok----tFileDelete
You would just have to pass the current path of the tFileList in the tRunJob_2.

Re: [resolved] Wait for Multiple Component Oks

Anonymous — Thu, 19 Jan 2012 14:02:25 GMT

Thank you! The second option worked like a charm. I had thought about the first option, but in the event that one of the files failed, none of them would be removed. The subjob worked perfectly though. Thanks again!