Unlock a world of possibilities! Login now and discover the exclusive benefits awaiting you.
Hi,
I would like to know why the bigdata components don't have runif trigger. I have tried to use a tjava component and checked if it has runif. It didnt work. I would like to trigger joblets from a bigdata job based on the inputs from context variables.
Hello,
RunIf and OnComponentOK triggers are not supported in Big Data job designers.
This limitation is valid for M/R and Spark jobs and is due to the way Big Data jobs are working and due to their distributed context : Big Data execution engines evaluate the Subjobs as a whole and not by component like Standard/DI job is doing, meaning that several components can be merged and processed in the same step by Spark (ex: if you do tExtractJson->tFilterCols->tFilterRow, this will be executed by Spark as one single map operation). As such, we can not guarantee that an individual component respects RunIf / OnComponentOK triggers (as this is not the granularity in the hand of the execution engine), and so these 2 triggers are not active in these job designers.
Please have a look at this related jira link:https://jira.talendforge.org/browse/TBD-4401
Best regards
Sabrina