topic Re: [resolved] Dynamic data source in Talend Studio

[resolved] Dynamic data source

Anonymous — Sat, 16 Nov 2024 10:47:56 GMT

Hi,
I have followed this post: http://stackoverflow.com/questions/30363304/how-can-i-install-postgresqljdbc-to-work-in-karaf-osgi
and setup a data source properly, however, I found there are some issues when I try to use dynamic data source in my job.
Here is the scenario:
I have a Route (cHTTP) which calls one Talend job based on certain input parameter. In this Talend job, it loads data source alias based on input parameters from a property file. Of course, there are a set of data source alias are configured in this property file and it could be changed anytime as long as data source is configured in Runtime.
However, I found if "Data Source alias" field in tPostgresqlConnection has to be configured with a fixed value, otherwise Route job will give me ClassNotFound exception which shows Route cannot find my Talend job class. (I am using TOS 6.0)
If tPostgresqlConnection cannot use dynamic data source setting, can I fetch data sources which are available in Karaf container and how to do it?

Re: [resolved] Dynamic data source

Anonymous — Thu, 18 Feb 2016 05:37:38 GMT

Hi,
Could you please paste the post you followed up into forum directly?
We can not check the url you posted.
In addition, what's the ESB build version you are using?
Best regards
Sabrina

Re: [resolved] Dynamic data source

Anonymous — Thu, 18 Feb 2016 05:57:44 GMT

Hi Sabrina,
The post is steps to config a postgresql data source in Talend Runtime by executing below Karaf shell scripts:

karaf@root()> feature:repo-add mvn:org.ops4j.pax.jdbc/pax-jdbc-features/0.7.0/xml/features
karaf@root()>feature:install  pax-jdbc-spec
karaf@root()>feature:install transaction
karaf@root()>bundle:install -s mvn:org.postgresql/postgresql/9.4-1200-jdbc41
karaf@root()> service:list org.osgi.service.jdbc.DataSourceFactory
karaf@root()>feature:install  pax-jdbc-config
karaf@root()>feature:install  pax-jdbc-pool-dbcp2
Then create a cfg file to give all database setup under KARAF_HOME/etc/org.ops4j.datasource-companymanager.cfg

After all steps, i can see my datasources:

karaf@trun()> service:list javax.sql.DataSource

----------------------
 databaseName = MobileCRM
 dataSourceName = MobileCRMDS
 felix.fileinstall.filename = file:/var/ESBSE/container/etc/org.ops4j.datasource-MobileCRMDS.cfg
 osgi.jdbc.driver.name = PostgreSQL JDBC Driver-pool-xa
 osgi.jndi.service.name = MobileCRMDS
 password = mobilecrmuser
 portNumber = 5432
 serverName = 192.168.100.121
 service.factoryPid = org.ops4j.datasource
 service.pid = org.ops4j.datasource.1bfab764-f3e3-4a3d-a1f8-fcc56774aa6a
 user = mobilecrmuser
 service.id = 605
 service.bundleid = 306
 service.scope = singleton
Provided by :
 OPS4J Pax JDBC Config (306)

----------------------
 databaseName = WeChat
 dataSourceName = WeChatDS
 felix.fileinstall.filename = file:/var/ESBSE/container/etc/org.ops4j.datasource-WeChatDS.cfg
 osgi.jdbc.driver.name = PostgreSQL JDBC Driver-pool-xa
 osgi.jndi.service.name = WeChatDS
 password = postgres
 portNumber = 5432
 serverName = 192.168.100.121
 service.factoryPid = org.ops4j.datasource
 service.pid = org.ops4j.datasource.a34df05b-a4d3-452a-8a54-6bd964669a25
 user = postgres
 service.id = 606
 service.bundleid = 306
 service.scope = singleton
Provided by :
 OPS4J Pax JDBC Config (306)

And I am working on TOS 6.0 community.

Re: [resolved] Dynamic data source

Anonymous — Thu, 18 Feb 2016 06:06:44 GMT

Data Source alias defined in blueprint thats why cannot configure usual way using context/variable.

Re: [resolved] Dynamic data source

Anonymous — Thu, 18 Feb 2016 06:32:43 GMT

Hi aLeXx
I am not getting you....
Data source alias is defined in blueprint, that is correct. This is how to define a data source, isn't?
But I am trying to access data source from my talend job's tpostgresqlconnection with a dynamic data source name (refer to a String variable), which gives me exceptions in Runtime.
And even I set Data source alias field in tpostgresqlconnection with simple string operation: " MobileCRM"+"DS" , same ClassNotFind exception occurred.

Re: [resolved] Dynamic data source

Anonymous — Sat, 20 Feb 2016 05:24:05 GMT

Resolved with walk around approach: https://www.talendforge.org/forum/viewtopic.php?pid=178256#p178256

Re: [resolved] Dynamic data source

Anonymous — Wed, 13 Jul 2016 20:03:21 GMT

Hi seanzhang1212,
The link you put in is not working. Do you remember what the fix was. I currently have the same issue and only a static value is currently being accepted. If I set a context variable or a globalMap it is using the same for datasource lookup instead of the value.
I have 3 oracle data sources setup. The incoming payload has the data source name and so want the data source alias to be set dynamically based on the incoming value.
Thanks