Do not input private or sensitive data. View Qlik Privacy & Cookie Policy.
Skip to main content

Announcements
Join us in Bucharest on Sept 18th for Qlik's AI Reality Tour! Register Now
cancel
Showing results for 
Search instead for 
Did you mean: 
Anonymous
Not applicable

Importing data from RDBMS to HDFS using SQOOP in talend

Hi,
I'm using TOS for Bigdata. I wanted to fetch table from MySQL to HDFS using SQOOP component.
My job looks like this : tLibraryLoad >> tSqoopImport
When i run this job its giving the error "tool.ImportTool: Imported Failed: java.net.UnknownHostException: myHost".
Eventhough i changed myHost to ip, but still the same error.
If i fetch the data from Mysql to hdfs without using SQOOP in seperate job, working fine.
Here i have used : tMySqlInput >> tHDFSOutput
I don't know whats going wrong when i use tSqoopImport. I'm properly giving the NamenodeURI and JobTracker host.
So where i'm going wrong. In Hadoop lib i have added "mysql-connector-java-5.1.16-bin.jar" and "sqoop-1.4.3-cdh4.6.0.jar" jars as well.
I have attached the screen shot of the job.
Thanks,
ShreeCS
0683p000009MDd2.png
Labels (3)
1 Reply
Anonymous
Not applicable
Author

Hi,
Is there any updates on the above mentioned issue.
I have copied both mySQL and sqoop jar in the $HADOOP_HOME/lib directory. Still facing the same issue.
Error :
Starting job rdbms_sqoop at 12:42 09/04/2014.

14/04/09 12:42:25 WARN conf.Configuration: fs.default.name is deprecated. Instead, use fs.defaultFS
connecting to socket on port 3513
connected
14/04/09 12:42:25 WARN sqoop.ConnFactory: $SQOOP_CONF_DIR has not been set in the environment. Cannot check for additional configuration.
14/04/09 12:42:25 INFO manager.MySQLManager: Preparing to use a MySQL streaming resultset.
14/04/09 12:42:25 INFO tool.CodeGenTool: Beginning code generation
14/04/09 12:42:26 INFO manager.SqlManager: Executing SQL statement: SELECT t.* FROM `orders` AS t LIMIT 1
14/04/09 12:42:26 INFO manager.SqlManager: Executing SQL statement: SELECT t.* FROM `orders` AS t LIMIT 1
14/04/09 12:42:26 INFO orm.CompilationManager: $HADOOP_MAPRED_HOME is not set
Note: \tmp\sqoop-Appladmin\compile\f5fc28709ceb16cf63d9020e0f0d11da\orders.java uses or overrides a deprecated API.
Note: Recompile with -Xlint:deprecation for details.
14/04/09 12:42:28 INFO orm.CompilationManager: Writing jar file: \tmp\sqoop-Appladmin\compile\f5fc28709ceb16cf63d9020e0f0d11da\orders.jar
14/04/09 12:42:28 WARN manager.MySQLManager: It looks like you are importing from mysql.
14/04/09 12:42:28 WARN manager.MySQLManager: This transfer can be faster! Use the --direct
14/04/09 12:42:28 WARN manager.MySQLManager: option to exercise a MySQL-specific fast path.
14/04/09 12:42:28 INFO manager.MySQLManager: Setting zero DATETIME behavior to convertToNull (mysql)
14/04/09 12:42:28 INFO mapreduce.ImportJobBase: Beginning import of orders
14/04/09 12:42:28 WARN mapreduce.JobBase: SQOOP_HOME is unset. May not be able to find all job dependencies.
14/04/09 12:42:29 ERROR tool.ImportTool: Imported Failed: java.net.UnknownHostException: cldx-433-377
Exception in component tSqoopImport_1
java.lang.Exception: The Sqoop import job has failed
at newproject.rdbms_sqoop_0_1.rdbms_sqoop.tSqoopImport_1Process(rdbms_sqoop.java:416)
at newproject.rdbms_sqoop_0_1.rdbms_sqoop.tLibraryLoad_1Process(rdbms_sqoop.java:334)
at newproject.rdbms_sqoop_0_1.rdbms_sqoop.runJobInTOS(rdbms_sqoop.java:636)
at newproject.rdbms_sqoop_0_1.rdbms_sqoop.main(rdbms_sqoop.java:502)
disconnected
Job rdbms_sqoop ended at 12:42 09/04/2014.

Thanks,
ShreeCS