topic Re: How to set and use globalVariable/context variable in Big Data Batch job in Talend Studio

How to set and use globalVariable/context variable in Big Data Batch job

Anonymous — Sat, 16 Nov 2024 08:54:39 GMT

Hi All,

I am not able to set globalVariable/context variable in Big Data Batch job. While, I can do so in standard job.

Is there any other way to set and use globalVariable/context?

Re: How to set and use globalVariable/context variable in Big Data Batch job

Anonymous — Thu, 28 Dec 2017 08:08:51 GMT

Hello,

For bigdata batch job, the talend recommendation is to use a DI job 'launcher' that loads the "dynamic" context.

So far, there is also no tContextLoad component for Big Data Batch and Spark Streaming.

Best regards

Sabrina

Re: How to set and use globalVariable/context variable in Big Data Batch job

Anonymous — Thu, 28 Dec 2017 08:13:40 GMT

Thanks Sabrina.

We were able to set context variable from DI job and pass it to Spark job.

But we are not able to reset that variable during runtime in spark job.

For e.g. if we want to capture status of certain operation in spark job, we
are not able to set variable as "success" or "failed".