Error whit metadata when i try to execute a job: tsqoopimport

Anonymous — Sat, 16 Nov 2024 08:20:09 GMT

Hi guys.

I'm trying to execute a job . This job is very simple: is a component tsqoopimport.

I have to export data from a SQL Server to Hive. But when a execute the job, i have the follown error:

[ERROR]: org.apache.sqoop.Sqoop - Got exception running Sqoop: org.kitesdk.data.DatasetIOException: Cannot access descriptor location: hdfs://SERVER/.metadata
at org.apache.sqoop.manager.SQLServerManager.importQuery(SQLServerManager.java:405)
at org.apache.sqoop.tool.ImportTool.importTable(ImportTool.java:509)
at org.apache.sqoop.tool.ImportTool.run(ImportTool.java:615)
at org.apache.sqoop.Sqoop.run(Sqoop.java:143)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:179)
at local_project.prueba_dos_0_1.prueba_dos.tSqoopImport_2Process(prueba_dos.java:422)
at local_project.prueba_dos_0_1.prueba_dos.runJobInTOS(prueba_dos.java:701)
at local_project.prueba_dos_0_1.prueba_dos.main(prueba_dos.java:558)
Caused by: java.net.ConnectException: Call From DESKTOP-ALQ/ to SERVER:8020 failed on connection exception: java.net.ConnectException: Connection refused: no further information; For more details see: http://wiki.apache.org/hadoop/ConnectionRefused
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
[statistics] disconnected
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
at org.apache.hadoop.net.NetUtils.wrapWithMessage(NetUtils.java:791)
at org.apache.hadoop.net.NetUtils.wrapException(NetUtils.java:731)
at org.apache.hadoop.ipc.Client.call(Client.java:1475)
at org.apache.hadoop.ipc.Client.call(Client.java:1408)
at org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:230)
at com.sun.proxy.$Proxy7.getFileInfo(Unknown Source)
at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolTranslatorPB.getFileInfo(ClientNamenodeProtocolTranslatorPB.java:762)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:256)
at org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:104)
at com.sun.proxy.$Proxy8.getFileInfo(Unknown Source)
at org.apache.hadoop.hdfs.DFSClient.getFileInfo(DFSClient.java:2102)
at org.apache.hadoop.hdfs.DistributedFileSystem$19.doCall(DistributedFileSystem.java:1215)
at org.apache.hadoop.hdfs.DistributedFileSystem$19.doCall(DistributedFileSystem.java:1211)
at org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolver.java:81)
at org.apache.hadoop.hdfs.DistributedFileSystem.getFileStatus(DistributedFileSystem.java:1211)
at org.apache.hadoop.fs.FileSystem.exists(FileSystem.java:1412)
at org.kitesdk.data.spi.filesystem.FileSystemMetadataProvider.checkExists(FileSystemMetadataProvider.java:561)
... 18 more
Caused by: java.net.ConnectException: Connection refused: no further information
at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:717)
at org.apache.hadoop.net.SocketIOWithTimeout.connect(SocketIOWithTimeout.java:206)
at org.apache.hadoop.net.NetUtils.connect(NetUtils.java:530)
at org.apache.hadoop.net.NetUtils.connect(NetUtils.java:494)
at org.apache.hadoop.ipc.Client$Connection.setupConnection(Client.java:614)
at org.apache.hadoop.ipc.Client$Connection.setupIOstreams(Client.java:713)
at org.apache.hadoop.ipc.Client$Connection.access$2900(Client.java:375)
at org.apache.hadoop.ipc.Client.getConnection(Client.java:1524)
at org.apache.hadoop.ipc.Client.call(Client.java:1447)
... 36 more
Exception in component tSqoopImport_2
java.lang.Exception: The Sqoop import job has failed. Please check the logs.
at local_project.prueba_dos_0_1.prueba_dos.tSqoopImport_2Process(prueba_dos.java:427)
at local_project.prueba_dos_0_1.prueba_dos.runJobInTOS(prueba_dos.java:701)
at local_project.prueba_dos_0_1.prueba_dos.main(prueba_dos.java:558)

I understand is a error with the metadata, but i don't know how to resolve. Thank you for your help!

Re: Error whit metadata when i try to execute a job: tsqoopimport

Anonymous — Sat, 28 Apr 2018 11:07:37 GMT

Hello,

tSqoopImport calls Sqoop to transfer data from a relational database management system (RDBMS) such as MySQL or Oracle into the Hadoop Distributed File System (HDFS).

For your hive server, you can create an external table into hive which points out a HDFS location or use thiveload, thiverow component.

Best regards

Sabrina

Re: Error whit metadata when i try to execute a job: tsqoopimport

Anonymous — Mon, 30 Apr 2018 13:30:25 GMT

Thank you Sabrina.

I was trying with a thiveload component but i have a problem: I loaded data in csv format , i need to load data in parquet format. The thiveload component does not have this option. Any ideas that how load data in parquet format with Talend?

load.JPG

Re: Error whit metadata when i try to execute a job: tsqoopimport

Anonymous — Tue, 01 May 2018 00:04:26 GMT

HELLO,

It seems issue with your Hadoop and Sqoop's configuration.Sqoop component uses MapReduce,so you need to specify classpath variable HADOOP_HOME and HADOOP_CONF_DIR and SQOOP_HOME and SQOOP_CONF_DIR(in your .bashrc file in ubuntu).
Please try it and inform if resolved.

hope this was helpfull!

Re: Error whit metadata when i try to execute a job: tsqoopimport

Anonymous — Wed, 02 May 2018 08:08:57 GMT

Hello,

Are you able to load files into the HDFS first? And then you can use Hive to interpret such files as database files.

Best regards

Sabrina

Re: Error whit metadata when i try to execute a job: tsqoopimport

Anonymous — Wed, 02 May 2018 12:50:11 GMT

Firts i loaded files into HDFS and then i used the thiveload to insert in the hive's database . Thank you Sabrina

topic Error whit metadata when i try to execute a job: tsqoopimport in Talend Studio

Error whit metadata when i try to execute a job: tsqoopimport

Re: Error whit metadata when i try to execute a job: tsqoopimport

Re: Error whit metadata when i try to execute a job: tsqoopimport

Re: Error whit metadata when i try to execute a job: tsqoopimport

Re: Error whit metadata when i try to execute a job: tsqoopimport

Re: Error whit metadata when i try to execute a job: tsqoopimport