topic Re: How to properly stop a tRabbitMQInput in an ETL Job in Talend Studio

How to properly stop a tRabbitMQInput in an ETL Job

FPellissier1625558358 — Wed, 31 May 2023 06:40:55 GMT

Hello,

In the documentation of the tRabbitMQInput component ( here ), we can read the following limitation :

Normally, this component works in the streaming mode and will not stop unless an error or an exception occurs. So, when you add this component to a Job, make sure that the Job has a mechanism that can interrupt the execution of this component.

What could be such a mechanism ?

A tJava with a "break" command doesn't work ( compilation error : "the break argument cannot be used outside of a loop..." )
A tDie ? It's seem quite extreme
A tJava with a System.exit ? Even more extreme as it kills the JVM
Another solution ?

Re: How to properly stop a tRabbitMQInput in an ETL Job

jlolling — Wed, 31 May 2023 09:03:28 GMT

Nearby... you can use tJavaRow within the flow and add here some code with an if condition and use break to to the flow. Please take care all input columns are handover to the output schema.

In fact Talend has currently no mechanism to gracefully shutdown a job.

Re: How to properly stop a tRabbitMQInput in an ETL Job

Anonymous — Wed, 31 May 2023 10:11:13 GMT

Define a condition based on the data retrieved from queue on tJavaRow, use 'break' command to stop the component execution once the condition is met. eg:

tRabbitMQInput--main--tJavaRow--main--tLogRow

|onsubjobok

other components

on tJavaRow:

output_row.columnName=input_row.columName;

if(input_row.columnName.equals("talend")){

break;

}

Can you try?

Using Die and check 'exit JVM immediatel' will kill the JVM and stop the job execution.

Regards

Shong

Re: How to properly stop a tRabbitMQInput in an ETL Job

RChauvin — Tue, 06 Jun 2023 07:42:49 GMT

Hello @Shicong Hong and @Jan Lolling

Thanks for your reply

To give you more details, we are trying to change the initial behavior of the tRabbitMQInput component, so that it is not used as a stream but as a classic row component (like a tDBInput for example).

So, this solution is not possible for us, cause we need to use tRabbitMQAck component to acknowledge messages one by one.

And this component does not accept row input, only onComponentOk/onSubJobOk or iterate. And as you know, we can't use break without row input

Also, if we had a clearer solution to cleanly stop a job at a chosen time without crashing the parent job (like a tDie or a system.exit, that would be ideal for us. We can indeed use if conditions to decide whether to continue processing or not, but it is a solution of last resort

Re: How to properly stop a tRabbitMQInput in an ETL Job

jlolling — Tue, 06 Jun 2023 11:03:37 GMT

You can use the component tFlowToIterate to convert the flow into a iterate trigger.

Re: How to properly stop a tRabbitMQInput in an ETL Job

RChauvin — Tue, 06 Jun 2023 11:26:04 GMT

Sûre, I know, but you can't use the break in an iterate, cause it's not the same loop as row