topic How to integrate MSSQL with Hive in Talend Studio

How to integrate MSSQL with Hive

Anonymous — Sat, 16 Nov 2024 10:51:01 GMT

Hi all, i am new developer in Talend, also learning Hadoop.
i have to come up with an ETL in Talend that reads from MSSQL and inserts this information into a Hive table.
As i am new, i dont know which components to use in my job, i have been able to test the connections to hive and sql but now i have to load the data in the hive table. Help please!

Re: How to integrate MSSQL with Hive

Anonymous — Mon, 25 Jan 2016 05:14:01 GMT

Hi,
You can create an external table into hive which points out a HDFS location. Then, you can use the tSqoopImport in order to import the Mysql table into HDFS (at the location you defined in your hive table creation).
Then your data will be available using Hive.
Let us know if it doesn't work for you.
Best regards
Sabrina

Re: How to integrate MSSQL with Hive

Anonymous — Tue, 26 Jan 2016 16:02:28 GMT

Thanks Sabrina! I was try that! but i also wonder if instead of using tSqoopImport component i could use a MSSQL connection and query the databases and store the result as a text file. I ask because the queries i need to execute are complex.
I appreciate your opinion!!
Thanks!!
Rosana

Re: How to integrate MSSQL with Hive

Anonymous — Wed, 27 Jan 2016 03:50:15 GMT

Hi,
Here is a component TalendHelpCenter:tHiveLoad which connects to a given Hive database and copies or moves data into an existing Hive table or a directory you specify.
Best regards
Sabrina

Re: How to integrate MSSQL with Hive

Anonymous — Tue, 29 May 2018 06:44:49 GMT

Can we replace SQLSERVER/MYSQL with HADOOP. Let's say I have created logic to load the data from source to SQLSERVER/MYSQL and later my customer wants to replace SQLSERVER/MYSQL database with HADOOP. Is it possible in Talend? If yes, what are all the limitations