topic how to load a csv file into hive temp table in Talend Studio

how to load a csv file into hive temp table

JaneYu — Sat, 16 Nov 2024 08:01:27 GMT

I need to load csv file into hive tables so I can write complicated logic in the hive query instead of expressions in the Talend.

Do I have to follow these steps? I did not test it yet.

1.tHiveCreateTable can be used to create a temp table

2.tFileInputDelimited (csv file)-> tHDFSOutput --store the csv in the HDFS system

3. then tHiveLoad --load the hdfs file to the hive table

4. use tHiveInput and get the data from the temp table

5. at the end of talend job, use tHiveRow to drop the temp table

Thank you.

Jane

Jesperrekuh — Mon, 02 Jul 2018 23:40:59 GMT

If its not heavy duty production processes, suggest using WebHDFS REST api to upload data.
No packages, no jars, just need cUrl or tRestClient.

JaneYu — Tue, 03 Jul 2018 14:31:19 GMT

After loading the csv to the temp hive table, it needs to join the other hive tables.

So I prefer to complete everything in Talend. Thank you