My suggestion could be used as a general solution. Such logics might be developed in a modular and cascading way maybe in outsourced sub-routines and in each application loaded per include-variables and general called or with some specialized parameter.
The more applications exists and the higher the complexity and by the requirement of a permanent availability of the environment you will need some own efforts to check the connections, network/storage, sources and the data and implement a more or less sophisticated error-handling. Further needed is a carefully monitoring of the task- and document-logging because you need to know the errors before you could fetch them with any logics.
In your case your issues may more come from too many parallel tasks which may try to access the same sources at the same time and blocking each other and/or the applied settings aren't really suitable for your scenario - in regard to the number of parallel tasks, how they are queued, with which timeouts and similar things …
- Marcus
