I use actually only one method which has been proven to be stable and scalable: Parallel execution of the Iterate trigger.
My job design is mostly separated in 2 jobs:
First job which proceed a piece of work (file, country, day etc.) and I call this job a worker job.
Second job detects the work to be done and iterate (enable here the parallelisation in the advanced settings of the Iterator) through the work items and use tRunJob to call the worker. The worker gets its work by context variables.
This has a lot of advantages:
1. you can easily test the worker (set for all necessary input context variables a prompt)
2. the amount of running worker job will be kept constantly by the engine
3. the work progress can easily be monitored
4. if you use the option independent child job in the tRunJob component you avoid memory leaks
5. This design works also in the Open Studio.
