tparallelize subjobs using same log file and statistics file

[vidya821]

Hi All,

Need suggestion here using tparallelize component

calling two subjobs in parallel using tparallelize component.

Can i use same log file and stat file in both the subjobs that are being executed parallely ?

tparallel ----->subjob1 (log file/stat file) using log and stat catcher

             ------>subjob2 (log file/stat file) using log and stat catcher

Is this possible and will it create any sort of job failure since the same log file/stat file is being used by the subjobs.

Thanks

[Anonymous]

Hi,

    You can use it but if you are using files to capture the result, there is a chance of failure if two processes are trying to write at same time. If you are using DB to capture the details, DB will automatically manage the conflict of writing two records at same time since the processing happens at row level and no file lock is happening.

Warm Regards,
Nikhil Thampi

Please appreciate our Talend community members by giving Kudos for sharing their time for your query. If your query is answered, please mark the topic as resolved 🙂

[vidya821]

Thanks Nikhil,

We are using append data feature...even then the job will fail ?

Thanks

[Anonymous · Accepted solution]

Hi,

    In that case, it should be fine.

    From my personal view, I am not a great fan of file related processing as there is always greater chances of getting data corrupted compared to DB. 

Warm Regards,
Nikhil Thampi

Please appreciate our Talend community members by giving Kudos for sharing their time for your query. If your query is answered, please mark the topic as resolved 🙂

[vidya821]

I understand that but log file data is mostly dynamic ..so using file instead of data