Skip to main content
cancel
Showing results for 
Search instead for 
Did you mean: 
Not applicable

Timing Reload task

hi,

i would like to know if there is a best practice documentation for timing the reload tasks

tnks!

1 Reply
Not applicable
Author

My manager has been asking me to put together some documentation on how I do it here....someday I'll get around to it .  Here is my "best practice".  Almost all of my loaders and reports are scheduled hourly.

Tier 1 (Extract) loaders run at the top of the hour.  I cascade loaders that run off the same database (run on success of previous task).  For 9 databases, I have 9 cascades of loaders. Since they are pulling from separate databases and are not computationally intensive (just extract), they can pretty much run in parallel.

Tier 2 (Transform) loaders run about 5-10 minutes after the Tier 1's finish.  Again, I cascade the loaders similarly to how I did the Tier 1.  I can see that my Tier1's finish around :10 after the hour; so I run them at :15 after the hour, allowing for a 5-minute buffer.  Since these are more computationally intensive, I limit how many tasks run in parallel by scheduling some at :15, :16, :17, etc...

Reports are scheduled about :30 after the hour.  These are grouped and cascaded by business group.

You an see my setup in this post:

Re: Any plans to deploy QDF now or in the future?

I know this isn't a formally documented best practice, but I hope it helps .

Cheers!