topic Re: Install and deploy talend jobs on remote Unix machine in Talend Studio

Install and deploy talend jobs on remote Unix machine

Anonymous — Thu, 08 Nov 2012 07:05:00 GMT

Hi,
I need to install talend open studio on a unix server which does not have UI.
I am assuming this will be done, just by kicking off the unix installable.
Please guide, if something else needs to be done in addition to this.
Deploying jobs is the area, where I am really concerned.
Till now, I was doing everything using UI on my local mac.
Is there a way in TOS to create jobs in local, export them and then reuse them in remote server.
Also, how to run these jobs.
Please provide inputs for achieving this.
Regards
Samya

Re: Install and deploy talend jobs on remote Unix machine

Anonymous — Thu, 08 Nov 2012 07:19:50 GMT

Hi,

Is there a way in TOS to create jobs in local, export them and then reuse them in remote server.
Also, how to run these jobs.

Talend open studio only support local job. You can reuse them on local machine not on remote server.Talend Enterprise studio can achieve that,.
Best regards
Sabrina

Re: Install and deploy talend jobs on remote Unix machine

Anonymous — Thu, 08 Nov 2012 07:54:01 GMT

Thanks for the prompt reply Sabrina.
Using TOS, can I export jobs from local , push them to remote machine and use there?
If this is possible, also tell how to run this job as the server does not have UI.
Also, how does Talend Enterprise Studio achieve remote execution of jobs?
Does it expose entire talend through a Url, where in we can deploy jobs, run them without logging to server.
Regards,
Sam

Re: Install and deploy talend jobs on remote Unix machine

Anonymous — Thu, 08 Nov 2012 08:43:47 GMT

Hi,

Using TOS, can I export jobs from local , push them to remote machine and use there

It can work, but in the studio(UI). There is only one way to export job by UI in Talend open studio without command line and you never use it which is closed.

Also, how does Talend Enterprise Studio achieve remote execution of jobs?
Does it expose entire talend through a Url, where in we can deploy jobs, run them without logging to server.

Not through a URL, we have TAC server and SVN repository. TIS can make connection with the server by Tomcat and all the job you created can be stored in server. When you remote, you can get connection with TAC server and SVN repository to get your own job. If you not logging to server, you can get your jobs which are on the server. The command line of TIS use to run some job without using component, such as, export job....
Hope that will help you.
Best regards
Sabrina

Re: Install and deploy talend jobs on remote Unix machine

Anonymous — Thu, 08 Nov 2012 10:59:09 GMT

Hi,

Is there a way in TOS to create jobs in local, export them and then reuse them in remote server.
Also, how to run these jobs.

Talend open studio only support local job. You can reuse them on local machine not on remote server.Talend Enterprise studio can achieve that,.
Best regards
Sabrina

Hello,

it's possible to export a job => Right click on the job, Export Job. You can select export type "Autonomous Job".
The archive generated contains all java lib dependencies, your jobs and a shell script.
You have just to put this archive on your Unix machine (you shoud have a SUN JVM 1.6 available on your server) and execute your job.
Regards,

Re: Install and deploy talend jobs on remote Unix machine

Anonymous — Thu, 08 Nov 2012 11:20:08 GMT

Does that mean, Talend jobs can be run on any machine just by exporting a job, even when talend is not installed on that machine..

Re: Install and deploy talend jobs on remote Unix machine

Anonymous — Fri, 09 Nov 2012 02:57:05 GMT

Hi,
Perhaps, i don't make my point clearly. In TOS, you can export your job and the archive generated contains all java lib dependencies, your jobs and a shell script. You can put it into any machine we support . Execute the job script, Directly! In WINDOWS, execution. BAT file, and in LINUX. Implementation. Sh file.
Best regards
Sabrina

Re: Install and deploy talend jobs on remote Unix machine

_AnonymousUser — Mon, 12 Nov 2012 20:36:04 GMT

Does that mean, Talend jobs can be run on any machine just by exporting a job, even when talend is not installed on that machine..

Yes - at least if Java is installed.

Re: Install and deploy talend jobs on remote Unix machine

Anonymous — Wed, 02 Sep 2015 11:07:43 GMT

Hi - I also looking for the solution to export the jobs developed in local machine to remote server. Please help me.
Developed job in Talend version - 5.6, in my local machine (windows).
In this version, I am not getting "Autonomous Job" while exporting. So I am using build type = "standalone job".
while trying to run the same in remote unix machine, its not running, and through error with jdbc connection. Is it possible to run the jobs in remote unix machine by exporting using "standalone job" type?
Thanks,
Robin

Re: Install and deploy talend jobs on remote Unix machine

Anonymous — Wed, 02 Sep 2015 11:10:26 GMT

One more clarification - Remote unix machine is the big data server. I am trying to run the java/Talend code. We have Java installed in this system.

Re: Install and deploy talend jobs on remote Unix machine

Anonymous — Sun, 06 Sep 2015 11:21:26 GMT

Hi Robin,
Talend Jobs support cross-platform execution. You can develop your Job on one machine, export the Job script, and then move it to another machine to execute without any additional configuration excepts the JDK installation.
Did you get your .bat file when exporting job on window? Does it work well on your local machine?

Best regards
Sabrina

Re: Install and deploy talend jobs on remote Unix machine

Anonymous — Mon, 07 Sep 2015 09:37:27 GMT

Hi Sabrina -
In local machine I am running Talend job from UI, connecting with Hortonwork Sandbox and it worked fine.
I got both .bat and .sh file in local machine, and exported entire build in to external machine Unix machine. And in external Unix machine, I ran .sh file.
Thanks,
Robin

Re: Install and deploy talend jobs on remote Unix machine

Anonymous — Tue, 08 Sep 2015 05:20:19 GMT

Hi Robin,
The job should be Ok if you can run .bat file on window machine successfully.

while trying to run the same in remote unix machine, its not running, and through error with jdbc connection.

Have you tried to repro this issue on your other jobs? Did you use context value in your current job with DB connection parameter?
Best regards
Sabrina

Re: Install and deploy talend jobs on remote Unix machine

Anonymous — Thu, 06 Apr 2017 14:50:48 GMT

Hi Sabrina,
As an extension to this, do you think I can set a remote Job server as default for the entire project or a user group instead of local host?
I was able to add a new JOb server through Preferences --> Talend --> Run/Remote and select the remote server before executing from Target Exec tab, but if I want the remote server to be selected by default for the entire project/team, can this is be achieved through a setting in TAC or something?
Best,
Mayank

Re: Install and deploy talend jobs on remote Unix machine

rduram — Mon, 27 Apr 2020 17:49:42 GMT

@xdshi wrote:
Hi,
Perhaps, i don't make my point clearly. In TOS, you can export your job and the archive generated contains all java lib dependencies, your jobs and a shell script. You can put it into any machine we support . Execute the job script, Directly! In WINDOWS, execution. BAT file, and in LINUX. Implementation. Sh file.
Best regards
Sabrina

Hi Sabrina - the problem I am experiencing with these steps is that my connection jar files to mysql (data source) and Redshift (data destination) are not the same on my laptop as I need them to be on my remote Linux server. On my laptop, I have a port forwarded through a jump host to reach the mysql database as well as another port forwarded through the same jump host to reach the Redshift DW. However, when I port the job to my Linux server, the connectivity path is direct (without the jump host) and uses the direct ports instead of my local forwarded ports. I am certain that the TOS Build Job command in the UI builds jar files for my local laptop forwarded ports resulting in 'Connection refused' communications errors (see below).

How do I create and specify different connection jar files to use on the Linux server?

******************************************************************************************************************************************************

bash Customer_Devices_run.sh
Exception in component tDBInput_2 (Customer_Devices)
com.mysql.cj.jdbc.exceptions.CommunicationsException: Communications link failure

The last packet sent successfully to the server was 0 milliseconds ago. The driver has not received any packets from the server.
at com.mysql.cj.jdbc.exceptions.SQLError.createCommunicationsException(SQLError.java:174)
at com.mysql.cj.jdbc.exceptions.SQLExceptionsMapping.translateException(SQLExceptionsMapping.java:64)
at com.mysql.cj.jdbc.ConnectionImpl.createNewIO(ConnectionImpl.java:832)
at com.mysql.cj.jdbc.ConnectionImpl.<init>(ConnectionImpl.java:456)
at com.mysql.cj.jdbc.ConnectionImpl.getInstance(ConnectionImpl.java:240)
at com.mysql.cj.jdbc.NonRegisteringDriver.connect(NonRegisteringDriver.java:207)
at java.sql/java.sql.DriverManager.getConnection(DriverManager.java:677)
at java.sql/java.sql.DriverManager.getConnection(DriverManager.java:228)
at insight_dw.customer_devices_0_1.Customer_Devices.tDBInput_2Process(Customer_Devices.java:7996)
at insight_dw.customer_devices_0_1.Customer_Devices.tDBInput_1Process(Customer_Devices.java:1791)
at insight_dw.customer_devices_0_1.Customer_Devices.runJobInTOS(Customer_Devices.java:15486)
at insight_dw.customer_devices_0_1.Customer_Devices.main(Customer_Devices.java:15328)
Caused by: com.mysql.cj.exceptions.CJCommunicationsException: Communications link failure

The last packet sent successfully to the server was 0 milliseconds ago. The driver has not received any packets from the server.
at java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
at java.base/jdk.internal.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.base/java.lang.reflect.Constructor.newInstance(Constructor.java:490)
at com.mysql.cj.exceptions.ExceptionFactory.createException(ExceptionFactory.java:61)
at com.mysql.cj.exceptions.ExceptionFactory.createException(ExceptionFactory.java:105)
at com.mysql.cj.exceptions.ExceptionFactory.createException(ExceptionFactory.java:151)
at com.mysql.cj.exceptions.ExceptionFactory.createCommunicationsException(ExceptionFactory.java:167)
at com.mysql.cj.protocol.a.NativeSocketConnection.connect(NativeSocketConnection.java:91)
at com.mysql.cj.NativeSession.connect(NativeSession.java:152)
at com.mysql.cj.jdbc.ConnectionImpl.connectOneTryOnly(ConnectionImpl.java:952)
at com.mysql.cj.jdbc.ConnectionImpl.createNewIO(ConnectionImpl.java:822)
... 9 more
Caused by: java.net.ConnectException: Connection refused (Connection refused)
at java.base/java.net.PlainSocketImpl.socketConnect(Native Method)
at java.base/java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:399)
at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:242)
at java.base/java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:224)
at java.base/java.net.SocksSocketImpl.connect(SocksSocketImpl.java:403)
at java.base/java.net.Socket.connect(Socket.java:609)
at com.mysql.cj.protocol.StandardSocketFactory.connect(StandardSocketFactory.java:173)
at com.mysql.cj.protocol.a.NativeSocketConnection.connect(NativeSocketConnection.java:65)
... 12 more

********************************************************************************************************************************************