SqoopImport issue: Query option not compatible with HCatalog
I found what I believe being a bug when using tSqoopImport: if you select the option 'Use query' you have to specify the destination directory. If you use HCatalog (see under 'Additional Arguments') and use --hcatalog-table, you get the error: "--hcatalog-table cannot be used --warehouse-dir or --target-dir options". The only way to solve the error is to use the option "Table Name" that automatically discards the Query option. Now, I'm running successfully a sqoop import via shell that uses the --query option and also HCatalog with the options --hcatalog-table; that cannot be done when using tSqoopImport in Talend. The correct behavior should be that when --hcatalog-table is used, the parameter --target-dir (pulled by 'Use query' option) should be ignored.
Is there any way --target-dir can be ignored when using the "Use query" option and the --hcatalog-table option under "Additional Arguments"?
I would like also to add that ORC is not listed as File Format option, and it should be (since we have parquet, avro, text, and sequence). The good news is that is you specify '--hcatalog-storage-stanza' argument as additional argument and add 'stored as orc tblproperties ("orc.compress"="ZLIB")', that would work. I suggest Talend to add ORC file format in the drop down.
Do we have a fix for this. Still we are facing this issue. Could you let us know how to use HCatalog and Query options together without using the target-dir option.
Below is the error we are facing:
"--hcatalog-table cannot be used --warehouse-dir or --target-dir options"
