topic Re: SqoopImport issue: Query option not compatible with HCatalog in Talend Studio

SqoopImport issue: Query option not compatible with HCatalog

Anonymous — Sat, 16 Nov 2024 10:21:19 GMT

I found what I believe being a bug when using tSqoopImport: if you select the option 'Use query' you have to specify the destination directory. If you use HCatalog (see under 'Additional Arguments') and use --hcatalog-table, you get the error: "--hcatalog-table cannot be used --warehouse-dir or --target-dir options". The only way to solve the error is to use the option "Table Name" that automatically discards the Query option. Now, I'm running successfully a sqoop import via shell that uses the --query option and also HCatalog with the options --hcatalog-table; that cannot be done when using tSqoopImport in Talend. The correct behavior should be that when --hcatalog-table is used, the parameter --target-dir (pulled by 'Use query' option) should be ignored.
Is there any way --target-dir can be ignored when using the "Use query" option and the --hcatalog-table option under "Additional Arguments"?
I would like also to add that ORC is not listed as File Format option, and it should be (since we have parquet, avro, text, and sequence). The good news is that is you specify '--hcatalog-storage-stanza' argument as additional argument and add 'stored as orc tblproperties ("orc.compress"="ZLIB")', that would work. I suggest Talend to add ORC file format in the drop down.

Re: SqoopImport issue: Query option not compatible with HCatalog

Anonymous — Mon, 10 Oct 2016 04:49:11 GMT

Hi,
Could you please indicate what's the build version you are using?
Best regards
Sabrina

Re: SqoopImport issue: Query option not compatible with HCatalog

_AnonymousUser — Tue, 11 Oct 2016 21:06:39 GMT

The build version is Talend Studio Real_time Big Data 6.2.1. Jira for the issue: TBD-4089 - HCatalog with tSqoopExport

Re: SqoopImport issue: Query option not compatible with HCatalog

ritikasharan — Wed, 20 Nov 2019 09:43:39 GMT

Do we have a fix for this. Still we are facing this issue. Could you let us know how to use HCatalog and Query options together without using the target-dir option.

Below is the error we are facing:

"--hcatalog-table cannot be used --warehouse-dir or --target-dir options"