Compose workflow schedule using cron expression

[NewbieQlik]

I have a requirement where I need to schedule one compose workflow from 5 AM EST to 10 PM EST, is it possible to build a cron expression which can be used to for my requirement.

I know the cron expression for compose is in UTC so I tried 0 0/2 9-1 * * ? will it work? Any valuable suggestions are greatly appreciated.

[Nanda_Ravindra]

Hello @NewbieQlik 

   What you have in the expression should work. But it's good to test it out. I see you have it scheduled to run every 2 minutes between 5 AM and 10 PM (EST). Is this what you want? If so, what type of job are you trying to run? FL or CDC job ?

[NewbieQlik]

Yes I wanted to run the wf every 2 mits from 5 AM to 10 PM EST and I am running the CDC job

[shashi_holla · Accepted solution]

Hi @NewbieQlik 

Try this 0 0/2 9,10,11,12 * * which should ideally run the job as follows:

Every 2 mins starting 9 AM till 9:58 and next schedule starts at 10 AM till 10:58 and last schedule starts at 12 PM and ends at 12:58 PM.

It's not mentioned that cron job time is in UTC and I'm assuming it will be Compose project settings (UTC/server time) but best to confirm based on the test results.

Thank you,

[Nanda_Ravindra · Accepted solution]

Hello @shashi_holla 

Please note, that the request here is to run the cronjob as an expression every 2 minutes between 5 AM and 10 PM (EST) and NOT from 10 AM to 12 PM.

The cronjob time in the UTC time zone will come up as a note when you select the scheduler run job as 'advanced' - see below

[shashi_holla]

Hi @Nanda_Ravindra 5AM EST is 9AM in UTC and one of the ways how we can do it. Hope this clarifies.

Thank you,

[Nanda_Ravindra]

Hello @NewbieQlik 

   Okay, as I mentioned earlier it's best to test this out. Before you schedule this job, I would recommend you set the scheduler to verbose. This way you can see the details in the Compose log for the scheduled job.

To set the verbose for the scheduler:

On the main Compose screen, click Management - Logs - Log Management. Scroll to the bottom and move the slide bar for Scheduler to Verbose, click OK to confirm, and click OK to save and close the Log Management screen.

Below is from my Compose log

"Compose.log"(11710,26):340 2022-04-19 16:56:33 [Scheduler ] [DEBUG] schedule 195 start time at 04/19/2022 20:56:33 +00:00 UTC
"Compose.log"(11711,26):340 2022-04-19 16:56:33 [Scheduler ] [DEBUG] schedule 195 is cron policy, Quartz expression is '0 0/2 9-1 * * ?'
"Compose.log"(11712,26):340 2022-04-19 16:56:33 [Scheduler ] [DEBUG] Job 195_Qjob has been scheduled
"Compose.log"(11713,26):340 2022-04-19 16:56:33 [Scheduler ] [DEBUG] next time is 04/19/2022 20:58:00 UTC
"Compose.log"(11714,26):340 2022-04-19 16:56:34 [Scheduler ] [DEBUG] next time is 04/19/2022 20:58:00 UTC

Let me know how it goes.

Thanks,

Nanda

[NewbieQlik · Accepted solution]

I have tested the cron expression 0 0/2 9-1 * * ? and it is working as expected.

[NewbieQlik]

I have one more requirement where I need to run another compose wf on BD9 (business day 9) every month. We have a table where we keep tract of whether the day is BD or CD (Calendar day) if the day is BD9 then we should trigger the full load workflow at 10.30 PM.  I don't think this is even possible using cron expression. Is there any way I can run the workflow based on the BD requirement? does this require any third party scheduling tool?

[NewbieQlik]

In my organization we are planning to use AWS Airflow as the enterprise scheduler. Can we use airflow for compose scheduling?