Issue with tSqoopInput ??

Bluemoon — Sat, 16 Nov 2024 09:29:45 GMT

I wanted to load MySQL data into Hadoop file system using sqoop in talend.. And i have also create a user and grand all the permission need..

GRANT ALL PRIVILEGES ON mydatabase.* to 'cloudera'@'xxx' IDENTIFIED BY 'cloudera';

Component Configuration

Starting job sample at 16:00 22/07/2017.

[statistics] connecting to socket on port 3656
[statistics] connected
[WARN ]: org.apache.sqoop.ConnFactory - $SQOOP_CONF_DIR has not been set in the environment. Cannot check for additional configuration.
[ERROR]: org.apache.sqoop.manager.SqlManager - Error executing statement: com.mysql.jdbc.exceptions.jdbc4.CommunicationsException: Communications link failure

The last packet sent successfully to the server was 0 milliseconds ago. The driver has not received any packets from the server.
com.mysql.jdbc.exceptions.jdbc4.CommunicationsException: Communications link failure

The last packet sent successfully to the server was 0 milliseconds ago. The driver has not received any packets from the server.
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
at com.mysql.jdbc.Util.handleNewInstance(Util.java:409)
at com.mysql.jdbc.SQLError.createCommunicationsException(SQLError.java:1127)
at com.mysql.jdbc.MysqlIO.<init>(MysqlIO.java:356)
at com.mysql.jdbc.ConnectionImpl.coreConnect(ConnectionImpl.java:2502)
at com.mysql.jdbc.ConnectionImpl.connectOneTryOnly(ConnectionImpl.java:2539)
at com.mysql.jdbc.ConnectionImpl.createNewIO(ConnectionImpl.java:2321)
at com.mysql.jdbc.ConnectionImpl.<init>(ConnectionImpl.java:832)
at com.mysql.jdbc.JDBC4Connection.<init>(JDBC4Connection.java:46)
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
at com.mysql.jdbc.Util.handleNewInstance(Util.java:409)
at com.mysql.jdbc.ConnectionImpl.getInstance(ConnectionImpl.java:417)
at com.mysql.jdbc.NonRegisteringDriver.connect(NonRegisteringDriver.java:344)
at java.sql.DriverManager.getConnection(DriverManager.java:664)
at java.sql.DriverManager.getConnection(DriverManager.java:247)
at org.apache.sqoop.manager.SqlManager.makeConnection(SqlManager.java:904)
at org.apache.sqoop.manager.GenericJdbcManager.getConnection(GenericJdbcManager.java:52)
at org.apache.sqoop.manager.SqlManager.execute(SqlManager.java:763)
at org.apache.sqoop.manager.SqlManager.execute(SqlManager.java:786)
at org.apache.sqoop.manager.SqlManager.getColumnInfoForRawQuery(SqlManager.java:289)
at org.apache.sqoop.manager.SqlManager.getColumnTypesForRawQuery(SqlManager.java:260)
at org.apache.sqoop.manager.SqlManager.getColumnTypes(SqlManager.java:246)
at org.apache.sqoop.manager.ConnManager.getColumnTypes(ConnManager.java:327)
at org.apache.sqoop.orm.ClassWriter.getColumnTypes(ClassWriter.java:1861)
at org.apache.sqoop.orm.ClassWriter.generate(ClassWriter.java:1661)
at org.apache.sqoop.tool.CodeGenTool.generateORM(CodeGenTool.java:107)
at org.apache.sqoop.tool.ImportTool.importTable(ImportTool.java:488)
at org.apache.sqoop.tool.ImportTool.run(ImportTool.java:615)
at org.apache.sqoop.Sqoop.run(Sqoop.java:143)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:179)
at bigdatademo.sample_0_1.sample.tSqoopImport_1Process(sample.java:591)
at bigdatademo.sample_0_1.sample.runJobInTOS(sample.java:917)
at bigdatademo.sample_0_1.sample.main(sample.java:733)
Caused by: java.net.ConnectException: Connection timed out: connect
at java.net.DualStackPlainSocketImpl.connect0(Native Method)
at java.net.DualStackPlainSocketImpl.socketConnect(DualStackPlainSocketImpl.java:79)
at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350)
at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206)
at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188)
at java.net.PlainSocketImpl.connect(PlainSocketImpl.java:172)
at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
at java.net.Socket.connect(Socket.java:589)
at java.net.Socket.connect(Socket.java:538)
at java.net.Socket.<init>(Socket.java:434)
at java.net.Socket.<init>(Socket.java:244)
at com.mysql.jdbc.StandardSocketFactory.connect(StandardSocketFactory.java:258)
at com.mysql.jdbc.MysqlIO.<init>(MysqlIO.java:306)
... 33 more
Exception in component tSqoopImport_1
java.lang.Exception: The Sqoop import job has failed. Please check the logs.
[ERROR]: org.apache.sqoop.tool.ImportTool - Encountered IOException running import job: java.io.IOException: No columns to generate for ClassWriter
at org.apache.sqoop.orm.ClassWriter.generate(ClassWriter.java:1667)
at org.apache.sqoop.tool.CodeGenTool.generateORM(CodeGenTool.java:107)
at org.apache.sqoop.tool.ImportTool.importTable(ImportTool.java:488)
at org.apache.sqoop.tool.ImportTool.run(ImportTool.java:615)
at org.apache.sqoop.Sqoop.run(Sqoop.java:143)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:179)
at bigdatademo.sample_0_1.sample.tSqoopImport_1Process(sample.java:591)
at bigdatademo.sample_0_1.sample.runJobInTOS(sample.java:917)
at bigdatademo.sample_0_1.sample.main(sample.java:733)

[FATAL]: bigdatademo.sample_0_1.sample - tSqoopImport_1 The Sqoop import job has failed. Please check the logs.
at bigdatademo.sample_0_1.sample.tSqoopImport_1Process(sample.java:595)
at bigdatademo.sample_0_1.sample.runJobInTOS(sample.java:917)
at bigdatademo.sample_0_1.sample.main(sample.java:733)
[statistics] disconnected
Job sample ended at 16:00 22/07/2017. [exit code=1]

Re: Issue with tSqoopInput ??

Anonymous — Sun, 23 Jul 2017 04:38:17 GMT

I've never been able to get this component to work. My workaround is to write a shell script and call it with tSSH.

That said, check that you have both the correct driver and the correct Java version: sqoop requires the same Java version on the client machine as on the cluster, right down to the minor version number. Your stack trace references driver/connection issues, so it's worthwhile to rule this out before you proceed.

Good luck!

David

Re: Issue with tSqoopInput ??

Bluemoon — Thu, 27 Jul 2017 21:51:55 GMT

I have checked the java version they are looking good.. Upon some research I found there will be some configuration need to be done on Cloudera manager to make the tsqoop component to work.
Please let me know any know those configuration??

topic Re: Issue with tSqoopInput ?? in Talend Studio

Issue with tSqoopInput ??

Re: Issue with tSqoopInput ??

Re: Issue with tSqoopInput ??