Dynamically define context variable values from flat file in Big data spark job

Anonymous — Sat, 16 Nov 2024 09:49:33 GMT

Hi,

I am trying to update context variable on run time from flat file. It is a big data spark job.

I am reading csv file where it has context variable and their value. With tJavaRow component, I am assigning context variable value. The same approach is working fine with Standard job but not with Big data job.

Where I am wrong in this approach? How we can assign values to context variables in Big data spark job during run time from flat file?

Thanks..

Re: Dynamically define context variable values from flat file in Big data spark job

Anonymous — Wed, 10 May 2017 08:41:34 GMT

Hi,

For BD batch jobs, the Talend recommendation is to use a DI job 'launcher' that loads the "dynamic" context.

So far, there is also no tContextLoad component for Big Data Batch and Spark Streaming.

Best regards

Sabrina

topic Re: Dynamically define context variable values from flat file in Big data spark job in Talend Studio

Dynamically define context variable values from flat file in Big data spark job

Re: Dynamically define context variable values from flat file in Big data spark job