topic tsqoopimport advance setting not working in Talend Studio

tsqoopimport advance setting not working

Anonymous — Sat, 16 Nov 2024 08:32:55 GMT

I'm trying to import the data from mysql to directly hive table using tsqoopimport component advance settings, but when i run the sqoop job fails and few warning messages as below,

[WARN ]: org.apache.sqoop.tool.BaseSqoopTool - It seems that you've specified at least one of following:
[WARN ]: org.apache.sqoop.tool.BaseSqoopTool - --hive-home
[WARN ]: org.apache.sqoop.tool.BaseSqoopTool - --hive-overwrite
[WARN ]: org.apache.sqoop.tool.BaseSqoopTool - --create-hive-table
[WARN ]: org.apache.sqoop.tool.BaseSqoopTool - --hive-table
[WARN ]: org.apache.sqoop.tool.BaseSqoopTool - --hive-partition-key
[WARN ]: org.apache.sqoop.tool.BaseSqoopTool - --hive-partition-value
[WARN ]: org.apache.sqoop.tool.BaseSqoopTool - --map-column-hive
[WARN ]: org.apache.sqoop.tool.BaseSqoopTool - Without specifying parameter --hive-import. Please note that
[WARN ]: org.apache.sqoop.tool.BaseSqoopTool - those arguments will not be used in this session. Either
[WARN ]: org.apache.sqoop.tool.BaseSqoopTool - specify --hive-import to apply them correctly or remove them.

///

Eventhough i have use hive.import(as i'm using JavaAPI mode) not sure what value to pass , hence left value

as ""(empty value).

Mysql is on different server and hive is on different server. I was able to import to hdfs using sqoop import but can't import directly to hive.

Please suggest me if i miss something.

Thanks,

Re: tsqoopimport advance setting not working

Anonymous — Mon, 26 Mar 2018 11:03:25 GMT

Hello,

You can create an external table into hive which points out a HDFS location. Then, you can use the tSqoopImport in order to import the Mysql table into HDFS (at the location you defined in your hive table creation).
Then your data will be available using Hive.
Let us know if it doesn't work for you.

Best regards

Sabrina

Re: tsqoopimport advance setting not working

Anonymous — Mon, 26 Mar 2018 17:02:25 GMT

Hi Thanks for your reply.

I tried external table solution while waiting for the reply to my post. It worked.

But looking for the solution , is there anyway we can directly import to hive managed table instead of external table concept.

Please confirm talend allow us to do direct import or is there any bug in talend advance settings for sqoop import to hive table.

Regards,

Kumar.

Re: tsqoopimport advance setting not working

Anonymous — Wed, 28 Mar 2018 04:18:14 GMT

Hello,

So far, talend doesn't allow you to do direct import by using sqoop.

Absolutely, you can combine a tMySQLInput and a tHiveRow to do that!

It would look like that:

tHiveConnection
|
|
v
tMySQLInput ----> tHiveRow
|
|
v
tHiveClose

Best regards

Sabrina