topic Re: tKafkaInput reset and starts from in Talend Studio

tKafkaInput reset and starts from

hildown — Sat, 16 Nov 2024 06:27:00 GMT

Is there a way to control tKafkaInput settings with context variables? For example, Advanced Settings -> properties?

I found there is a auto.offset.reset setting for 'New consumer group starts from', but I haven't found a similar setting that corresponds to the 'Reset offsets on consumer group' checkbox.

Adding more detail. On a first run if 'Reset offsets on consumer group' is unchecked and 'New consumer group starts from' is set to latest, I get no kafka data. So for a first run, I would need checked/beginning. Thereafter I want to run with unchecked/latest.

Re: tKafkaInput reset and starts from

Anonymous — Thu, 14 Mar 2019 09:24:51 GMT

Hello,

If we understand your requirement very well, do you want to select checkbox option via context variable?

So far, it is impossible to set the checkbox value at runtime, unless we change filed type from 'CHECK' TO 'TEXT'. Feel free to correct us if we misunderstand your requirement.

Best regards

Sabrina

Re: tKafkaInput reset and starts from

hildown — Thu, 14 Mar 2019 13:25:37 GMT

Correct, I'd like to select checkbox option via context variable.

Without that, we're left with some poor workarounds:

1) initial run of job should have all tkafkainput set to beginning; then modify job to set all tkafkainput to latest for future runs

2) or put if logic into flow - if firstrun follow path that has tkafkainput set to beginning, else follow path that has latest

Can you think of a better way?