Install and deploy talend jobs on remote Unix machine
Hi,
I need to install talend open studio on a unix server which does not have UI.
I am assuming this will be done, just by kicking off the unix installable.
Please guide, if something else needs to be done in addition to this.
Deploying jobs is the area, where I am really concerned.
Till now, I was doing everything using UI on my local mac.
Is there a way in TOS to create jobs in local, export them and then reuse them in remote server.
Also, how to run these jobs.
Please provide inputs for achieving this.
Regards
Samya
Hi Robin,
Talend Jobs support cross-platform execution. You can develop your Job on one machine, export the Job script, and then move it to another machine to execute without any additional configuration excepts the JDK installation.
Did you get your .bat file when exporting job on window? Does it work well on your local machine?
Hi Sabrina - In local machine I am running Talend job from UI, connecting with Hortonwork Sandbox and it worked fine. I got both .bat and .sh file in local machine, and exported entire build in to external machine Unix machine. And in external Unix machine, I ran .sh file. Thanks, Robin
Hi Robin,
The job should be Ok if you can run .bat file on window machine successfully.
while trying to run the same in remote unix machine, its not running, and through error with jdbc connection.
Have you tried to repro this issue on your other jobs? Did you use context value in your current job with DB connection parameter?
Best regards
Sabrina
Hi Sabrina,
As an extension to this, do you think I can set a remote Job server as default for the entire project or a user group instead of local host?
I was able to add a new JOb server through Preferences --> Talend --> Run/Remote and select the remote server before executing from Target Exec tab, but if I want the remote server to be selected by default for the entire project/team, can this is be achieved through a setting in TAC or something?
Best,
Mayank
@xdshi wrote:
Hi,
Perhaps, i don't make my point clearly. In TOS, you can export your job and the archive generated contains all java lib dependencies, your jobs and a shell script. You can put it into any machine we support . Execute the job script, Directly! In WINDOWS, execution. BAT file, and in LINUX. Implementation. Sh file.
Best regards
Sabrina
Hi Sabrina - the problem I am experiencing with these steps is that my connection jar files to mysql (data source) and Redshift (data destination) are not the same on my laptop as I need them to be on my remote Linux server. On my laptop, I have a port forwarded through a jump host to reach the mysql database as well as another port forwarded through the same jump host to reach the Redshift DW. However, when I port the job to my Linux server, the connectivity path is direct (without the jump host) and uses the direct ports instead of my local forwarded ports. I am certain that the TOS Build Job command in the UI builds jar files for my local laptop forwarded ports resulting in 'Connection refused' communications errors (see below).
How do I create and specify different connection jar files to use on the Linux server?
bash Customer_Devices_run.sh Exception in component tDBInput_2 (Customer_Devices) com.mysql.cj.jdbc.exceptions.CommunicationsException: Communications link failure
The last packet sent successfully to the server was 0 milliseconds ago. The driver has not received any packets from the server. at com.mysql.cj.jdbc.exceptions.SQLError.createCommunicationsException(SQLError.java:174) at com.mysql.cj.jdbc.exceptions.SQLExceptionsMapping.translateException(SQLExceptionsMapping.java:64) at com.mysql.cj.jdbc.ConnectionImpl.createNewIO(ConnectionImpl.java:832) at com.mysql.cj.jdbc.ConnectionImpl.<init>(ConnectionImpl.java:456) at com.mysql.cj.jdbc.ConnectionImpl.getInstance(ConnectionImpl.java:240) at com.mysql.cj.jdbc.NonRegisteringDriver.connect(NonRegisteringDriver.java:207) at java.sql/java.sql.DriverManager.getConnection(DriverManager.java:677) at java.sql/java.sql.DriverManager.getConnection(DriverManager.java:228) at insight_dw.customer_devices_0_1.Customer_Devices.tDBInput_2Process(Customer_Devices.java:7996) at insight_dw.customer_devices_0_1.Customer_Devices.tDBInput_1Process(Customer_Devices.java:1791) at insight_dw.customer_devices_0_1.Customer_Devices.runJobInTOS(Customer_Devices.java:15486) at insight_dw.customer_devices_0_1.Customer_Devices.main(Customer_Devices.java:15328) Caused by: com.mysql.cj.exceptions.CJCommunicationsException: Communications link failure
The last packet sent successfully to the server was 0 milliseconds ago. The driver has not received any packets from the server. at java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) at java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) at java.base/jdk.internal.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.base/java.lang.reflect.Constructor.newInstance(Constructor.java:490) at com.mysql.cj.exceptions.ExceptionFactory.createException(ExceptionFactory.java:61) at com.mysql.cj.exceptions.ExceptionFactory.createException(ExceptionFactory.java:105) at com.mysql.cj.exceptions.ExceptionFactory.createException(ExceptionFactory.java:151) at com.mysql.cj.exceptions.ExceptionFactory.createCommunicationsException(ExceptionFactory.java:167) at com.mysql.cj.protocol.a.NativeSocketConnection.connect(NativeSocketConnection.java:91) at com.mysql.cj.NativeSession.connect(NativeSession.java:152) at com.mysql.cj.jdbc.ConnectionImpl.connectOneTryOnly(ConnectionImpl.java:952) at com.mysql.cj.jdbc.ConnectionImpl.createNewIO(ConnectionImpl.java:822) ... 9 more Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:399) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:242) at java.base/java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:224) at java.base/java.net.SocksSocketImpl.connect(SocksSocketImpl.java:403) at java.base/java.net.Socket.connect(Socket.java:609) at com.mysql.cj.protocol.StandardSocketFactory.connect(StandardSocketFactory.java:173) at com.mysql.cj.protocol.a.NativeSocketConnection.connect(NativeSocketConnection.java:65) ... 12 more
