topic extract the metadata from the dynamic type field in Talend Studio

extract the metadata from the dynamic type field

Shafee — Sat, 16 Nov 2024 08:03:43 GMT

Need to load data and metadata from source is Oracle and target is Hive
Daily receive a file which have list of table names(daily the list count changes and even new table also added)
want to design a job which iterates on each table in file and load the metadata and data from source to target.

If my source(oracle) have int or date type or any data type then same type should be in target(hive) in the same job.

want load metadata and data in same job

Re: extract the metadata from the dynamic type field

Anonymous — Wed, 01 Aug 2018 06:37:33 GMT

Hello,

You need to load files in oracle source into the HDFS first and then you can use Hive to interpret such files as database files.

tSqoopImport calls Sqoop to transfer data from a relational database management system (RDBMS) such as Oracle into the Hadoop Distributed File System (HDFS).

For your hive server, you can create an external table into hive which points out a HDFS location or use thiveload, thiverow component.

Best regards

Sabrina

Re: extract the metadata from the dynamic type field

Shafee — Mon, 19 Nov 2018 13:51:42 GMT

thanks for help, will try with this solution