Streaming table changes to Kafka

[Anonymous]

Hi,

I'm trying to capture SQL Server table changes in real-time and send them to a Kafka topic.

I know that I can use a CDC table and connect it to Kafka directly (CDC table -> Kafka), but is there another alternative using Talend (something like CDC table -> Talend -> Kafka, etc)?

I've tried using Talend's CDC feature, but it seems that it only works for batch jobs and not real-time... Do correct me if I'm wrong though.

Regards,

Veronica

[vapukov]

Hi,

any solution would be not a real-time (unless you not design sending a message direct from triggers), all will be - loop by schedule or infinite loop

so, you could run Talend job in loop - 1 sec delay or less

or you could check in internet SQL CDC to Kafka and find few other solution for read SQL Server CDC tables, but it still will be not a real real-time

[Anonymous]

Hi,

Yes, I am aware that I can create real-time streams from SQL Server directly to Kafka, but I was wondering whether it's possible to do it in a more user-friendly way with Talend.

Thanks for the input!

[vapukov]

as I know - not, all what you can do it schedule job in a loop (schedule or design Job as infinite loop)

with some other tools it same - you must define query interval