Ability for Compose to start a databricks cluster prior to scheduled run
We have a CDC task in C4DL for a databricks project that is scheduled to run daily at 4am. At 4am the task fails due to Cluster being unavailable. We are requesting that compose start up the cluster and then begin processing the CDC task instead of failing. This is similar to how replicate task with databricks target starts the cluster when changes are detected in source system that need to be applied to the target.