With Talend ESB, how to override a route context parameter on runtime ?

[Anonymous]

Hello everyone, and thanks for reading me, I try to deploy a route built with Talend Open Studio For Esb, 6.5, on windows server 2016 Datacenter
To do so I built the route, then manually dropped the file on the container/deploy folder.

Then using client.bat I tried to set some configuration parameters to override values defined in the context of the project (in my case the name of an sqs-queue), but it does not work.

First of all, is it the right approach to configure parameters for a Route ?

Things I have done :

• unzipped the zip file to disk D
• followed the stepd to install the wrapper as a windows service
• client.bat >
  • config:edit MyRouteName
  • config: property-set myProp myValue
  • config:update
  • refresh (also tried bundle:refresh)
  • stop MyRouteName
  • start MyRouteName

Did I missed something ?

Is there a way to check what parameters are actually applied to the Route ? Is the .cfg file read watched continuously for changes (I also tried to edit it manually) ?

Thanks
Florian

[Anonymous]

Florian - 

I'm relatively new at this, but here are some thoughts: 

1. Yes, the files are read dynamically and the changes should be instantaneous. If you look at the Apache Karaf documentation on configs, you can see that the two methods (config: or editing the file) are interchangeable. In addition, "If you provide a configuration PID that doesn’t exist yet, Apache Karaf will create a new configuration (and so a new configuration file) automatically." 
2. That said, I'm not sure you can overwrite a specific context variable within your context file, but you can switch the context using these commands. So if you had a different contexts established in the Studio for each of your sqs-queues (i.e. sqs-queue1, sqs-queue2, sqs-queue3, etc.), you could switch between them by: 

config:edit MyRouteName
config: property-set context sqs-queue1
config:update

You could then verify the changes with: 

config:list "(service.pid=MyRouteName)"

... I think. 

[dleerob]

I am having a similar problem where context variables on ESB are not updating, using Talend 7.1.1.

If I deploy a new service via the ESB Conductor which uses a context named "qc", and it has a variable called "mdm_port" with value "8080", it works the first time deploying.

On the ESB Runtime server, I can see the value of each context variable here: "/opt/Talend-7.1.1/esb/container/data/cache/org.eclipse.osgi/9/data/config/{MyService}/talendcontext/qc.config"

If I then change a context variable value of "mdm_port" in Talend Studio to "8081", and publish to Nexus, and then undeploy and re-deploy or even delete and re-add via ESB Conductor on TAC, the value does not change on the ESB Server file "/opt/Talend-7.1.1/esb/container/data/cache/org.eclipse.osgi/9/data/config/{MyService}/talendcontext/qc.config". It still shows the "mdm_port" as "8080".

If I then manually try and change the context values on the ESB runtime server in that qc.config file, my job/service is still using the old value. Even after restarting karaf etc.

So basically, once you have deployed a service for the first time with a context and context variables defined, you cannot change them ? Anyone know how I can update context variable values on my Talend ESB Runtime ?

I have opened a support ticket with Talend Support but they are so slow to respond and it has been three days of battling with this already.

[davidferguson]

Hi dleerob,

I'm currently evaluating 7.1.1 and was hoping this issue had been resolved in this release - did you get an update on your case that you can share?

Thanks,

David.

[dleerob]

Hi

Unfortunately I am still waiting for a solution or feedback.  The case is currently still "under investigation".

I took me a considerable amount of time and effort to get them to understand the problem. I ended up having to create the simplest job possible which returns the context variable values on a REST call with 14 screenshots showing deployment, updating the variables, and re-deployment where the updated variable values aren't showing.

I'm a little concerned that something like this hasn't been picked up by their QC team and fixed. It's surely something effecting anyone using contexts on the ESB.

Anyway the only workaround I have found is to override the values in the ESB conductor and deploy to the ESB.  However, even those overridden values have an issue in that the quotes around string values (which you are supposed to include) dont get stripped out on the ESB like they usually do and so are actually included in the variable value. So now you need to override the value without being surrounded by quotes.

[davidferguson]

I'm assuming you're deploying a REST Service Job as an OSGi bundle in Karaf. If so, there's a bug ticket for this exact case: https://jira.talendforge.org/browse/TESB-11530 It's apparently been open since 2013 and has a lot of Support Cases associated with it... You might want to have a read of it and see if you can associate your support case with it too.

So expecting context variables used in the URL of a REST Web Service Job to change on a configuration refresh is wishful thinking unfortunately (I've not tested SOAP Jobs or Routes for this use case)... However, I have been testing the 7.1.1 release and am happy to note that the use of contexts in other places within Routes do get updated - also when Routes are provisioned via the bundled Talend Nexus (there were issues with this in 6.4.1). We've stopped using TAC/Conductor for ESB as there were too many issues with it. Instead, we use the feature:add-repo, config:* etc. commands directly in karaf to administrate the runtime.

EDIT: I've just re-read the bug link I posted - you may have left a comment there already, pehaps?