Unlock a world of possibilities! Login now and discover the exclusive benefits awaiting you.
Hi all,
Can we schedule Replica execution jobs according to dependency like we do for dags in airflow?
Thanks,
Replicate itself only offers simple time based scheduling, think Linux cron jobs. No dependencies no status checks or even finish checks for prior jobs.
Scheduling is typically only useful for Full-Load only tasks. Does the design call for that?
The dominant Replicate usage and it's claim to fame is near real-time ongoing Replication. Set and forget!
It's tasks are data streams, not the usual programming building blocks which might conditionally run.
There is a command line (repctl) and an (enterprise manager provided) RestApi which could be used to start and stop tasks based on time and conditions but those are just primitives and you'd have to provide a lot of work to for example declare success or failure or still running for prior steps.
Good luck!
DAG/airflow sample explanation: https://docs.astronomer.io/learn/dag