topic Re: [resolved] How can I export tables from MySQL to Hive/Hadoop? in Talend Studio

[resolved] How can I export tables from MySQL to Hive/Hadoop?

Anonymous — Sat, 16 Nov 2024 12:37:26 GMT

How can I export tables from MySQL to Hive/Hadoop? Which components will help? like tHiveConnection,tHiveRow or tHiveClose?
Thank you for your replies.

Re: [resolved] How can I export tables from MySQL to Hive/Hadoop?

Anonymous — Mon, 17 Oct 2011 20:05:47 GMT

Hi,
Absolutely, you can combine a tMySQLInput and a tHiveRow to do that!
It would look like that:
tHiveConnection
|
|
v
tMySQLInput ----> tHiveRow
|
|
v
tHiveClose

Re: [resolved] How can I export tables from MySQL to Hive/Hadoop?

_AnonymousUser — Wed, 26 Oct 2011 18:26:25 GMT

Could you provide more detail on how to use the tHiveRow component here? You can't do a single-row insert in Hive/HQL.
Thanks.
---Paul

Re: [resolved] How can I export tables from MySQL to Hive/Hadoop?

Anonymous — Wed, 23 Nov 2011 03:53:35 GMT

Could you provide more detail on how to use the tHiveRow component here? You can't do a single-row insert in Hive/HQL.
Thanks.
---Paul

vgalopin is right,but this method is inefficient,data is imported one by one.Actually, I recommend "load overwrite" command used in tHiveRow's query like this "load data local inpath '"+context.hive_file_dir+"t_test.txt' overwrite into table t_test"

Re: [resolved] How can I export tables from MySQL to Hive/Hadoop?

Anonymous — Wed, 23 Nov 2011 04:03:11 GMT

Through using this tool(Talend),I find it supports hive/hadoop not very well.Free 4.2.3 version only provide three components,which are tHiveConnection,tHiveRow and tHiveClose.

Re: [resolved] How can I export tables from MySQL to Hive/Hadoop?

Anonymous — Mon, 28 Nov 2011 19:17:24 GMT

Please note that a tHiveInput component is planned to get released on 5.0.