Best Practice for component/subjob exclusion based on context?
Hello Talend Community,
i am wondering if there are any best practices out there and what is your experience regarding "context aware job execution"?
What i mean by this:
For example we are using Talend for Data Integration to load from some sources for which i cannot easily trigger output for DEV/QA Environemnts in order to develop and test Talend Jobs.
This causes me to deactivate certain aspects of the job while developing and runing the job in the Designer console (e.g. cleanup tasks like "delete source files when job execution was successful" ) - or i need to fake the source data into the DEV source if i want to check the built and exported job for DEV context. This becomes quite clunky.
What would be really handy to do: Omit certain components or entire subjobs based on the context a job is executed in. I found so far that this will not work with the standard "Activation/Deactivation" functionality in the designer - but maybe you have any ideas how to achive this on runtime?
You can easily create different routes through a job using the RunIf links and context variables. Instead of linking subjobs with OnSubJobOk links, maybe link the subjob to a tJava (dummy) component, then link the tJava to the next subjob with RunIF links.
You can easily create different routes through a job using the RunIf links and context variables. Instead of linking subjobs with OnSubJobOk links, maybe link the subjob to a tJava (dummy) component, then link the tJava to the next subjob with RunIF links.
Another suggestion might be to use the Dynamic Job Invocation in a tRunJob component that gets set at runtime to invoke, say, the DEV version of the cleanup you mention (that doesn't do anything).
I would however suggest caution when using any approach like this as you are really not going to be able to properly test your integrations for how they will behave in a production environment.
I will take a shot on the "RunIf" solution. I guess that just means that each job / subjob should be designed in a way that the Tasks which are only supposed to run in some contexts (e.g. "Prod") are always at the end of a given route with a "RunIf" Statement leading into them.
