Importing data from RDBMS to HDFS using SQOOP in talend

Anonymous — Sat, 16 Nov 2024 11:41:30 GMT

Hi,
I'm using TOS for Bigdata. I wanted to fetch table from MySQL to HDFS using SQOOP component.
My job looks like this : tLibraryLoad >> tSqoopImport
When i run this job its giving the error "tool.ImportTool: Imported Failed: java.net.UnknownHostException: myHost".
Eventhough i changed myHost to ip, but still the same error.
If i fetch the data from Mysql to hdfs without using SQOOP in seperate job, working fine.
Here i have used : tMySqlInput >> tHDFSOutput
I don't know whats going wrong when i use tSqoopImport. I'm properly giving the NamenodeURI and JobTracker host.
So where i'm going wrong. In Hadoop lib i have added "mysql-connector-java-5.1.16-bin.jar" and "sqoop-1.4.3-cdh4.6.0.jar" jars as well.
I have attached the screen shot of the job.
Thanks,
ShreeCS

Re: Importing data from RDBMS to HDFS using SQOOP in talend

Anonymous — Wed, 09 Apr 2014 08:13:21 GMT

Hi,
Is there any updates on the above mentioned issue.
I have copied both mySQL and sqoop jar in the $HADOOP_HOME/lib directory. Still facing the same issue.
Error :
Starting job rdbms_sqoop at 12:42 09/04/2014.

14/04/09 12:42:25 WARN conf.Configuration: fs.default.name is deprecated. Instead, use fs.defaultFS
connecting to socket on port 3513
connected
14/04/09 12:42:25 WARN sqoop.ConnFactory: $SQOOP_CONF_DIR has not been set in the environment. Cannot check for additional configuration.
14/04/09 12:42:25 INFO manager.MySQLManager: Preparing to use a MySQL streaming resultset.
14/04/09 12:42:25 INFO tool.CodeGenTool: Beginning code generation
14/04/09 12:42:26 INFO manager.SqlManager: Executing SQL statement: SELECT t.* FROM `orders` AS t LIMIT 1
14/04/09 12:42:26 INFO manager.SqlManager: Executing SQL statement: SELECT t.* FROM `orders` AS t LIMIT 1
14/04/09 12:42:26 INFO orm.CompilationManager: $HADOOP_MAPRED_HOME is not set
Note: \tmp\sqoop-Appladmin\compile\f5fc28709ceb16cf63d9020e0f0d11da\orders.java uses or overrides a deprecated API.
Note: Recompile with -Xlint:deprecation for details.
14/04/09 12:42:28 INFO orm.CompilationManager: Writing jar file: \tmp\sqoop-Appladmin\compile\f5fc28709ceb16cf63d9020e0f0d11da\orders.jar
14/04/09 12:42:28 WARN manager.MySQLManager: It looks like you are importing from mysql.
14/04/09 12:42:28 WARN manager.MySQLManager: This transfer can be faster! Use the --direct
14/04/09 12:42:28 WARN manager.MySQLManager: option to exercise a MySQL-specific fast path.
14/04/09 12:42:28 INFO manager.MySQLManager: Setting zero DATETIME behavior to convertToNull (mysql)
14/04/09 12:42:28 INFO mapreduce.ImportJobBase: Beginning import of orders
14/04/09 12:42:28 WARN mapreduce.JobBase: SQOOP_HOME is unset. May not be able to find all job dependencies.
14/04/09 12:42:29 ERROR tool.ImportTool: Imported Failed: java.net.UnknownHostException: cldx-433-377
Exception in component tSqoopImport_1
java.lang.Exception: The Sqoop import job has failed
at newproject.rdbms_sqoop_0_1.rdbms_sqoop.tSqoopImport_1Process(rdbms_sqoop.java:416)
at newproject.rdbms_sqoop_0_1.rdbms_sqoop.tLibraryLoad_1Process(rdbms_sqoop.java:334)
at newproject.rdbms_sqoop_0_1.rdbms_sqoop.runJobInTOS(rdbms_sqoop.java:636)
at newproject.rdbms_sqoop_0_1.rdbms_sqoop.main(rdbms_sqoop.java:502)
disconnected
Job rdbms_sqoop ended at 12:42 09/04/2014.

Thanks,
ShreeCS

topic Re: Importing data from RDBMS to HDFS using SQOOP in talend in Talend Studio

Importing data from RDBMS to HDFS using SQOOP in talend

Re: Importing data from RDBMS to HDFS using SQOOP in talend