Skip to main content
Announcements
A fresh, new look for the Data Integration & Quality forums and navigation! Read more about what's changed.
cancel
Showing results for 
Search instead for 
Did you mean: 
Anonymous
Not applicable

Unable to create hive external table with avro.schema.url

Hello Experts,

 

I'm trying to create Hive external table using talend data fabric 7.1.1 but although I gave avro.schema.url the table creation is throwing an error "at least one column must be specified for the table". I have .avsc file kept at the avro.schema.url location.

I'm using thivecreatetable component with create external table box checked, built in schema is empty 'cuz I want to retrieve it from .avsc file located at avro.schema.url. 

 

Table Properties                        value

"avro.schema.url"                    "hdfs://path/filename.avsc"

 

Could somebody help please? 

Labels (3)
2 Replies
Anonymous
Not applicable
Author

Hello,

We currently don't have a way in the Studio to automate getting the Avro Schema from a file, especially on HDFS.

To do it manually, please have a look at avro-tools (see http://www.michael-noll.com/blog/2013/03/17/reading-and-writing-avro-files-from-the-command-line/ for a good tutorial). 

There's a "getschema" command, but it requires the avro file to be on the local filesystem.  This is probably the best way to do it for one or two tables.

Please refer to this new feature jira issue:https://jira.talendforge.org/browse/TBD-5870

Best regards

Sabrina

Anonymous
Not applicable
Author

Hello,

Even if I get schema retrieved from avro file, my question is how do I create external table based on the .avsc file, which is not working with tHiveCreateTable component? what is the best way to create external table based on .avsc file?