1 Reply Latest reply: Feb 19, 2014 9:30 AM by Chadd McNicholas RSS

    Timing Reload task

      hi,

       

      i would like to know if there is a best practice documentation for timing the reload tasks

       

      tnks!

        • Re: Timing Reload task

          My manager has been asking me to put together some documentation on how I do it here....someday I'll get around to it .  Here is my "best practice".  Almost all of my loaders and reports are scheduled hourly.

           

          Tier 1 (Extract) loaders run at the top of the hour.  I cascade loaders that run off the same database (run on success of previous task).  For 9 databases, I have 9 cascades of loaders. Since they are pulling from separate databases and are not computationally intensive (just extract), they can pretty much run in parallel.

           

          Tier 2 (Transform) loaders run about 5-10 minutes after the Tier 1's finish.  Again, I cascade the loaders similarly to how I did the Tier 1.  I can see that my Tier1's finish around :10 after the hour; so I run them at :15 after the hour, allowing for a 5-minute buffer.  Since these are more computationally intensive, I limit how many tasks run in parallel by scheduling some at :15, :16, :17, etc...

           

          Reports are scheduled about :30 after the hour.  These are grouped and cascaded by business group.

           

          You an see my setup in this post:

          Re: Any plans to deploy QDF now or in the future?

           

          I know this isn't a formally documented best practice, but I hope it helps .

           

          Cheers!