topic Re: Error while running sqoop using talend in Talend Studio

Error while running sqoop using talend

Anonymous — Sat, 16 Nov 2024 11:43:53 GMT

I am using Talend 5.3.1 open studio for big data.I am trying to import data from SQL server to HDFS using sqoop
but when i am running the job i get the following error
"ERROR security.UserGroupInformation: PriviledgedActionException "
How to solve this error?
I have face this same problem when i was running MR jobs using eclipse so when i used the below code the error was resolved.
"UserGroupInformation ugi = UserGroupInformation.createRemoteUser("hduser");"
So is there any option to create remote user.

Re: Error while running sqoop using talend

Anonymous — Tue, 04 Mar 2014 22:39:25 GMT

Hello,
Don't you have a more detailed exception?
What is the confirguration of your Sqoop component? Which Sqoop component do you use?
Cheers,

Re: Error while running sqoop using talend

Anonymous — Wed, 05 Mar 2014 03:19:18 GMT

I am using CDH4.5 Sqoop component the the detailed error is
"14/03/03 17:22:02 ERROR security.UserGroupInformation: PriviledgedActionException as:296922 (auth IMPLE) cause:java.net.ConnectException: Call From INFPH01332/localhost to localhost:8021 failed on connection exception: java.net.ConnectException: Connection refused: no further information; For more details see: http://wiki.apache.org/hadoop/ConnectionRefused
14/03/03 17:22:02 ERROR tool.ImportTool: Encountered IOException running import job: java.net.ConnectException: Call From INFPH01332/localhost to localhost:8021 failed on connection exception: java.net.ConnectException: Connection refused: no further information; For more details see: http://wiki.apache.org/hadoop/ConnectionRefused
at org.apache.hadoop.net.NetUtils.wrapException(NetUtils.java:727)
at org.apache.hadoop.ipc.Client.call(Client.java:1165)
at org.apache.hadoop.ipc.WritableRpcEngine$Invoker.invoke(WritableRpcEngine.java:224)
at org.apache.hadoop.mapred.$Proxy11.getStagingAreaDir(Unknown Source)
at org.apache.hadoop.mapred.JobClient.getStagingAreaDir(JobClient.java:1223)
at org.apache.hadoop.mapreduce.JobSubmissionFiles.getStagingDir(JobSubmissionFiles.java:102)
at org.apache.hadoop.mapred.JobClient$2.run(JobClient.java:850)
at org.apache.hadoop.mapred.JobClient$2.run(JobClient.java:844)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:396)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1232)
at org.apache.hadoop.mapred.JobClient.submitJobInternal(JobClient.java:844)
at org.apache.hadoop.mapreduce.Job.submit(Job.java:481)
at org.apache.hadoop.mapreduce.Job.waitForCompletion(Job.java:511)
at org.apache.sqoop.mapreduce.ImportJobBase.doSubmitJob(ImportJobBase.java:163)
at org.apache.sqoop.mapreduce.ImportJobBase.runJob(ImportJobBase.java:141)
at org.apache.sqoop.mapreduce.ImportJobBase.runImport(ImportJobBase.java:211)
at org.apache.sqoop.manager.SqlManager.importTable(SqlManager.java:465)
at org.apache.sqoop.manager.SQLServerManager.importTable(SQLServerManager.java:99)
at org.apache.sqoop.tool.ImportTool.importTable(ImportTool.java:403)
at org.apache.sqoop.tool.ImportTool.run(ImportTool.java:476)
at org.apache.sqoop.Sqoop.run(Sqoop.java:145)
at sampleproj.sqoopconnect_0_1.SqoopConnect.tSqoopImport_2Process(SqoopConnect.java:641)
at sampleproj.sqoopconnect_0_1.SqoopConnect.tLibraryLoad_2Process(SqoopConnect.java:563)
at sampleproj.sqoopconnect_0_1.SqoopConnect.runJobInTOS(SqoopConnect.java:877)
at sampleproj.sqoopconnect_0_1.SqoopConnect.main(SqoopConnect.java:743)
Caused by: java.net.ConnectException: Connection refused: no further information
at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:567)
at org.apache.hadoop.net.SocketIOWithTimeout.connect(SocketIOWithTimeout.java:206)
at org.apache.hadoop.net.NetUtils.connect(NetUtils.java:524)
at org.apache.hadoop.net.NetUtils.connect(NetUtils.java:489)
at org.apache.hadoop.ipc.Client$Connection.setupConnection(Client.java:472)
at org.apache.hadoop.ipc.Client$Connection.setupIOstreams(Client.java:566)
at org.apache.hadoop.ipc.Client$Connection.access$2000(Client.java:215)
at org.apache.hadoop.ipc.Client.getConnection(Client.java:1271)
at org.apache.hadoop.ipc.Client.call(Client.java:1141)
... 24 more
Exception in component tSqoopImport_2
java.lang.Exception: The Sqoop import job has failed
at sampleproj.sqoopconnect_0_1.SqoopConnect.tSqoopImport_2Process(SqoopConnect.java:643)
at sampleproj.sqoopconnect_0_1.SqoopConnect.tLibraryLoad_2Process(SqoopConnect.java:563)
at sampleproj.sqoopconnect_0_1.SqoopConnect.runJobInTOS(SqoopConnect.java:877)
at sampleproj.sqoopconnect_0_1.SqoopConnect.main(SqoopConnect.java:743)"

Re: Error while running sqoop using talend

Anonymous — Wed, 05 Mar 2014 12:13:12 GMT

Hi,
Your Talend studio is installed on the same machine than the cluster?

Re: Error while running sqoop using talend

Anonymous — Wed, 05 Mar 2014 16:12:14 GMT

Hi,
I have recently resolved this issue with my sqoopimport in talend.
This issue is coming when your jobtracker is not able to communicate to namenode.
is your hadoop cluster a single node cluster or multiple node?
can you locate your core-site.xml file?. as i have installed cloudera manager my core-site.xml was existing in
/opt/cloudera/parcels/CDH-4.5.0-1.cdh4.5.0.p0.30/lib/hadoop-0.20-mapreduce/example-confs/conf.pseudo/core-site.xml
edit the above file. by default the fs.defaultname is always hdfs://localhost:8020.
change it to the ipaddress of your namenode.
<property>
<name>fs.default.name</name>
<value>hdfs://10.176.381.211:8020</value>
</property>
restart your hdfs from cloudera manager.
This should resolve the connection refused error and at least move you forward.
Regards
Gokul

Re: Error while running sqoop using talend

Anonymous — Thu, 06 Mar 2014 10:04:28 GMT

My Talend studio is installed on other machine than the cluster.
I have also edited the core-site.xml , replacing the localhost to ip- address

Re: Error while running sqoop using talend

Anonymous — Thu, 06 Mar 2014 18:52:06 GMT

Hi,
after the changes are you still getting the same error?
how you are executing your job?
can you use the oozie tab in talend to run the job on the hadoop cluster?
Thanks

Re: Error while running sqoop using talend

_AnonymousUser — Fri, 17 Feb 2017 12:14:05 GMT

Hi Team,
I am facing the below error when i am trying to sqoop data from MySQL DB to HDFS (Cloudera quickstart for cdh5.8) which is a single node cluster. Version is TOS_BD-V6.3.0. Had tried using both JAVA API and Commandline options but no luck. Connections with HDFS, Hive and MySQL DB are successful.
My job is as follows : tLibraryLoad --> tSqoopImport
When i use JAVA API : : org.apache.sqoop.tool.ImportTool - Encountered IOException running import job: java.io.IOException: Job status not available
When i use Command line :: java.io.IOException: Cannot run program "sqoop": CreateProcess error=2, The system cannot find the file specified

Re: Error while running sqoop using talend

Anonymous — Wed, 22 Feb 2017 05:08:43 GMT

Hi,
Your Talend studio is installed on the same machine than the cluste

I installed Hadoop 1.1.2 and Hive 0.9.0. I didn't set username and password when i installed Hive.What can i set in the username and password part?