topic Re: [resolved] Running Talend jobs parallelly in Talend Studio

[resolved] Running Talend jobs parallelly

Anonymous — Tue, 08 Jul 2014 10:43:53 GMT

Hi All,
I tried to run two sub jobs in parallel as shown in the image. Both the tOracleOutput components
are connecting to different tables in the same database(but they do not use a tOracleConnection).
Then I checked the 'Multi thread execution' option in the Run View of the job and started running the job.
Only one of the jobs ran successfully and the other failed with the following error. Does anyone has a solution for this?
Exception in component tOracleOutput_3
java.sql.SQLException: Invalid number format for port number
at oracle.jdbc.driver.SQLStateMapping.newSQLException(SQLStateMapping.java:70)
at oracle.jdbc.driver.DatabaseError.newSQLException(DatabaseError.java:131)
at oracle.jdbc.driver.DatabaseError.throwSqlException(DatabaseError.java:197)
at oracle.jdbc.driver.DatabaseError.throwSqlException(DatabaseError.java:525)
at oracle.jdbc.driver.T4CConnection.logon(T4CConnection.java:413)
at oracle.jdbc.driver.PhysicalConnection.<init>(PhysicalConnection.java:508)
at oracle.jdbc.driver.T4CConnection.<init>(T4CConnection.java:203)
at oracle.jdbc.driver.T4CDriverExtension.getConnection(T4CDriverExtension.java:33)
at oracle.jdbc.driver.OracleDriver.connect(OracleDriver.java:510)
at java.sql.DriverManager.getConnection(Unknown Source)
at java.sql.DriverManager.getConnection(Unknown Source)
at test.data_mig_demo_0_1.DATA_MIG_DEMO.tOracleInput_4Process(DATA_MIG_DEMO.java:5885)
at test.data_mig_demo_0_1.DATA_MIG_DEMO$3.run(DATA_MIG_DEMO.java:7176)
Caused by: oracle.net.ns.NetException: Invalid number format for port number
at oracle.net.resolver.AddrResolution.resolveSimple(AddrResolution.java:499)
at oracle.net.resolver.AddrResolution.resolveAndExecute(AddrResolution.java:396)
at oracle.net.ns.NSProtocol.establishConnection(NSProtocol.java:630)
at oracle.net.ns.NSProtocol.connect(NSProtocol.java:206)
at oracle.jdbc.driver.T4CConnection.connect(T4CConnection.java:966)
at oracle.jdbc.driver.T4CConnection.logon(T4CConnection.java:292)
... 8 more

Re: [resolved] Running Talend jobs parallelly

Anonymous — Wed, 09 Jul 2014 03:41:25 GMT

Hi,
Have you already checked your input parameter in tOracleOutput_3? Did you add port or is your port correct?
Best regards
Sabrina

Re: [resolved] Running Talend jobs parallelly

Anonymous — Wed, 09 Jul 2014 03:57:25 GMT

Yes, my port is correct and this works file when 'multi thread execution' is unchecked. I read the database configurations (including the port) from a property file and save it in globalMap and utilize them. I have attached my whole job.

Re: [resolved] Running Talend jobs parallelly

Anonymous — Wed, 09 Jul 2014 04:38:26 GMT

Hi,
Could you please use tlogrow instead of tOracleOutput_3 to see if this subjob works?
Best regards
Sabrina

Re: [resolved] Running Talend jobs parallelly

Anonymous — Wed, 09 Jul 2014 04:56:17 GMT

Hi Sabrina,
I tried it with tLogRow instead of tOracleOutput_3 and now it works (with 'Multi thread execution' enabled). I wonder how it happens. Could you please explain it for me?
Thanks,
Thivanka.

Re: [resolved] Running Talend jobs parallelly

Anonymous — Wed, 09 Jul 2014 08:16:14 GMT

hi all,
can we see what are parameters in your Oracle output (the errored one) ?
regards
laurent

Re: [resolved] Running Talend jobs parallelly

Anonymous — Wed, 09 Jul 2014 08:22:00 GMT

I have attached my basic settings and advanced settings.

Re: [resolved] Running Talend jobs parallelly

Anonymous — Wed, 09 Jul 2014 08:37:14 GMT

Hi,
From your screenshot, it seems you mask your oralce info with variable? Is it working well when you input DB parameter directly instead of using variable?
Have you checked component reference TalendHelpCenter:tSetGlobalVar
Did you press Ctrl Space bar on your keyboard and select the relevant global parameter? ((String)globalMap.get(?targetDBHost?))?
Best regards
Sabrina

Re: [resolved] Running Talend jobs parallelly

Anonymous — Wed, 09 Jul 2014 08:48:38 GMT

Hi Sabrina,
Yes, it worked parallely when I directly use the DB parameters instead using variable. I cant figure out why it does not work once I use the variables.
Yes, I pressed CTRL+SPACE and selected it.
To add that it works when I uncheck the 'Multi Thread Execution' option with variables in the configurations. The error comes only when I check that option.

Re: [resolved] Running Talend jobs parallelly

Anonymous — Wed, 09 Jul 2014 08:53:50 GMT

Hi,
Should it be ((String)globalMap.get(?targetDBHost?)) in variable list? Why there is missing start/end quote in your screenshot?
Best regards
Sabrina

Re: [resolved] Running Talend jobs parallelly

Anonymous — Wed, 09 Jul 2014 08:59:46 GMT

Hi Sabrina,
I don't think it matters. Since in Java, ((String)globalMap.get(?targetDBHost?)) and (String)globalMap.get(?targetDBHost?) results in the same output. If you see the generated Java code you can make sure that these two are identical.
Thanks,
Thivanka.

Re: [resolved] Running Talend jobs parallelly

Anonymous — Wed, 09 Jul 2014 09:00:13 GMT

Hi,
Could you please try to add another tSetGlobalVar_3 in your tOracleOutput_3 subjob with same setting(Host, port...)so that in case that tOracleOutput_3 can not get the global variable when the whole job is under the condition of Multi thread execution. (The 3 subjobs will be executed in a same time).
Best regards
Sabrina

Re: [resolved] Running Talend jobs parallelly

Anonymous — Wed, 09 Jul 2014 09:00:41 GMT

I think that your globalMap variables are still"empty" as your subjob start at the very first time of your job. (the green starting component tell me that)
Talend process begin & try to open connection to your Oracle DB before works with data.
link with onSubjobOk instead of a parallel execution.
regards
laurent
ps : Sabrina tell it better & faster than I

Re: [resolved] Running Talend jobs parallelly

Anonymous — Wed, 09 Jul 2014 09:47:16 GMT

Thanks Sabrina and Laurent.
You are correct. I tried removing the onSubjobOk link from tFileInputXML_2 to FINXML_ACTIVITY. Then both tOracleOutputs didnt work which means that when all the sub jobs starts running in parallel, the variables required for tOracleOutputs have not been initialized. So what I can do is to create a new tSetGlobalVar_3 and link the subjob with it.

Re: [resolved] Running Talend jobs parallelly

Anonymous — Wed, 09 Jul 2014 09:58:07 GMT

Hi asthiwanka,
If you want to mask your DB connection info, you can refer to tcontextload component reference with a scenario TalendHelpCenter: Reading data from different MySQL databases using dynamically loaded connection parameters.
Best regards
Sabrina