[resolved] No way to dynamically override dropdown component settings via parameters

[dan2]

Edited the title to help others in the future and marked the comment with the workaround.

Please see the screenshots below. I have a Joblet triggered by Component Ok in the main Job. The first step in the Joblet sets the parameters dynamically for a tS3Connection. The Debug log shows that the tS3Connection component is failing to read the new context. It should have updated to PROFILE_INHERIT_CREDENTIALS, but instead stays with the default even though the context was definitely updated by the tJava. How can I correct this?

 

 

 

 

I went ahead and output the full context upon failure of the tS3Connection component and can show that the context was updated, so tS3Connection seems to not be reading it. I also tried adding all variables as globals, but this did not work either.

[Anonymous]

This likely a timing issue. Can you show a screenshot of your job and a separate screenshot of your joblet that sets these values?

A joblet is essentially a collection of components that act is if they are part of the job you have used it in. My suspicion here is that your tS3Connection is instantiated before the context variables are set.

[dan2]

Hi @Richard Hall​:

GetSecretsAndDefaults_t Joblet is the 2nd step to run in the PreJob:

Step 1Step 2 (Joblet GetSecretsAndDefaults)

Step 3 (After TRIGGER_INPUT_1 is tJava3 which finds the user is not the Cloud user and sets the Local parameters via context variable changes)

Step 4 ( some unrelated setting of globals)

Step 5 (tS3Connection_1 <-- the issue. The parameter overrides are not getting picked up by this Component. I have also tried to set the parameters as global, but that did not work.)

Step 6 (on failure, we are able to see that the context variables were updated, but do not seem to have been picked up by tS3Connection_1)

Here are some larger pictures

Thanks again for your help!

[dan2]

@Richard Hall​ As a test, I also attempted to use tRowGenerater->tJavaRow->tContextLoad->tJava->tS3Connection. It loads the context variables ahead of the tS3Connection, but it still does not register the new parameter value. This feels like a bug..

 

 

[dan2]

@Richard Hall​ I went ahead and defined the Contexts directly in the Joblet to rule out an issue there and found that that did not even work. Oddly enough, printing the context immediately upon Joblet start comes back totally empty. I only am able to see the Context if I run context.synchronizeContext();, but that seems to only result in it being available in the prior tJava and tS3Connection is not affected. I believe tS3Connection sees no context from the beginning and does not care that a synchronize command occurred. To align with the documentation, I went ahead and put what I needed into a Context Group with the various context environments. I then dragged that Context Group into the Joblet. Talend appears to totally ignore the values and show blank context when the Joblet begins.

I then removed all context from the joblet and the main job and dragged the Context Group into the main job. The contexts still show up blank when I run it this way and the overrides are still not working.

[Anonymous]

Can you try this? After you have set your context variables in your tJava (straight after your Trigger_Input), use an OnSubJobOK link to your next component. I still think this is a timing issue. You connection component would be better placed in a subsequent SubJob.

[dan2]

@Richard Hall​ That unfortunately did not change the outcome. I'm beginning to think that the variables attached to the parameter in tS3Connection cannot be overridden due to a Talend code bug. Ideally, I do like the Context Group approach, but even hard-coding __CREDENTIAL_PROVIDER__ in the main job is not overriding tS3Connection.

 

[dan2]

Am I misunderstanding the meaning of "The variable attached to this parameter is:"__CREDENTIAL_PROVIDER__"? Am I not allowed to override component variables using environment context? If the settings cannot be changed dynamically in this manner, than I'd want to understand the best practice running the job across different environments that have different connection configs.

[Anonymous]

Context variables shouldn't really be set within a running job, unless you are using a tContextLoad at the beginning of the job. However, I have used this technique quite a lot. Sometimes I have had to use this code to ensure the values are set immediately.....

context.setProperty("{context name case sensitive}", {context value})

....my next suggestion was to try that. The other thing to try is using a tContextLoad component.

But, when changing values in a Job, it is arguably better to use the globalMap. Try my suggestions above. If they don't work, then the globalMap should. It is very difficult to identify if this is a bug without having access to your job. But from what I have tried in v8, I don't think this is a bug. I still think this a timing issue.

[dan2]

I took a step back and now am attempting this by adding the context per best practice, as a Context Group to the main Job, but that does not even work for this override. I have already attempted to set __CREDENTIAL_PROVIDER__ in the globalMap and using context.setPropertly. Same results. To be clear, __CREDENTIAL_PROVIDER__ is a dropdown in tS3Connection, so it's not like I can input java into the field to reference a global..