Unlock a world of possibilities! Login now and discover the exclusive benefits awaiting you.
hi,
i would like to know if there is a best practice documentation for timing the reload tasks
tnks!
My manager has been asking me to put together some documentation on how I do it here....someday I'll get around to it . Here is my "best practice". Almost all of my loaders and reports are scheduled hourly.
Tier 1 (Extract) loaders run at the top of the hour. I cascade loaders that run off the same database (run on success of previous task). For 9 databases, I have 9 cascades of loaders. Since they are pulling from separate databases and are not computationally intensive (just extract), they can pretty much run in parallel.
Tier 2 (Transform) loaders run about 5-10 minutes after the Tier 1's finish. Again, I cascade the loaders similarly to how I did the Tier 1. I can see that my Tier1's finish around :10 after the hour; so I run them at :15 after the hour, allowing for a 5-minute buffer. Since these are more computationally intensive, I limit how many tasks run in parallel by scheduling some at :15, :16, :17, etc...
Reports are scheduled about :30 after the hour. These are grouped and cascaded by business group.
You an see my setup in this post:
Re: Any plans to deploy QDF now or in the future?
I know this isn't a formally documented best practice, but I hope it helps .
Cheers!