Talend Studio CDC Job – How to Keep It Running Continuously?
Hi all,
I’ve created a CDC job in Talend Studio. It runs successfully, but then disconnects after completion.
I’m exploring ways to keep the job running continuously for several hours before disconnecting automatically.
I've considered scheduling it every 10–20 minutes, but I’d prefer an alternative solution if one exists. Has anyone implemented a more robust approach?
Talend CDC jobs are not designed to run continuously as long-running processes. By design, a CDC job connects to the source, captures the available changes, commits the offsets, and then disconnects.
Keeping a CDC job running for several hours is not recommended and can lead to unnecessary resource consumption (open connections, memory usage, locks).
Supported approaches are:
Scheduled execution (recommended): Run the CDC job at short intervals (for example every 5–20 minutes). Each execution processes incremental changes and exits cleanly.
Controlled re-execution: If near-real-time processing is required, implement a controlled loop with a wait/sleep mechanism, ensuring proper connection handling and error management.
Orchestration: Use Talend Administration Center / Talend Cloud scheduler or an external scheduler for monitoring and restartability.
Best practice: Talend CDC is intended for frequent, incremental executions, not continuous open connections.
