Multiple kafka consumers

DrGenious — Sat, 16 Nov 2024 00:14:48 GMT

I was wondering if there is a way to consume from kafka with multiple consumers without deploying multiple jobs, e.g. configuring tkafkainput to spawn multiple consumers or having multiple tkafkainput in a single job. Is ti possible?

Re: Multiple kafka consumers

Anonymous — Mon, 19 Apr 2021 11:42:09 GMT

Hello,

For Multiple Consumers, could you please take an example?

Multiple consumers are supported through a Consumer Group ID as specified in tKafkaInput where offset is managed through 3 settings;

* Reset offsets on consumer group

* New consumer group starts from { Beginning | Latest }

* Auto-commit offsets Interval

Best regards

Sabrina

topic Multiple kafka consumers in Talend Studio

Multiple kafka consumers

Re: Multiple kafka consumers