topic Re: Unable to create hive external table with avro.schema.url in Talend Studio

Unable to create hive external table with avro.schema.url

Anonymous — Sat, 16 Nov 2024 06:13:16 GMT

Hello Experts,

I'm trying to create Hive external table using talend data fabric 7.1.1 but although I gave avro.schema.url the table creation is throwing an error "at least one column must be specified for the table". I have .avsc file kept at the avro.schema.url location.

I'm using thivecreatetable component with create external table box checked, built in schema is empty 'cuz I want to retrieve it from .avsc file located at avro.schema.url.

Table Properties value

"avro.schema.url" "hdfs://path/filename.avsc"

Could somebody help please?

Re: Unable to create hive external table with avro.schema.url

Anonymous — Tue, 02 Apr 2019 07:44:12 GMT

Hello,

We currently don't have a way in the Studio to automate getting the Avro Schema from a file, especially on HDFS.

To do it manually, please have a look at avro-tools (see http://www.michael-noll.com/blog/2013/03/17/reading-and-writing-avro-files-from-the-command-line/ for a good tutorial).

There's a "getschema" command, but it requires the avro file to be on the local filesystem. This is probably the best way to do it for one or two tables.

Please refer to this new feature jira issue:https://jira.talendforge.org/browse/TBD-5870

Best regards

Sabrina

Re: Unable to create hive external table with avro.schema.url

Anonymous — Tue, 02 Apr 2019 16:22:38 GMT

Hello,

Even if I get schema retrieved from avro file, my question is how do I create external table based on the .avsc file, which is not working with tHiveCreateTable component? what is the best way to create external table based on .avsc file?