topic import metadata from talend bigdata to talend data catalog in Talend Data Catalog

import metadata from talend bigdata to talend data catalog

mshafeeq — Fri, 15 Nov 2024 22:05:51 GMT

hello all,

i need to know how to import metadata from talend bigdata to talend data catalog in steps ?

and there is anyway to import all data from the environment like hdfs,hive,impala and different sources to talend data catalog ?

thanks.

Re: import metadata from talend bigdata to talend data catalog

Anonymous — Thu, 23 Feb 2023 02:49:59 GMT

You could use this bridge in TDC to import Talend jobs to harvest the metadata in TDC.-Talend Data Integration - Import

https://help.talend.com/r/en-US/8.0/tdc_bridges/mirtalendimport?tocId=YzEFFiPLhZ_lhXpIcI~OCA

Below bridges can harvest HDFS, Hive, Impala.

-Apache Hadoop Distributed File System (HDFS Java API) - Import

================================================================

https://help.talend.com/r/en-US/8.0/tdc_bridges/mirapachehdfsimport-Apache Hadoop Hive Database (Hcatalog and Metastore via JDBC) - Import

==========================================================================

https://help.talend.com/r/en-US/8.0/tdc_bridges/mirapachehiveimport-Cloudera Impala Hadoop Hive Server - Import

=============================================

https://help.talend.com/r/en-US/8.0/tdc_bridges/mirclouderaimpalaimport

If you are using enterprise subscription license, I suggest you open tickets on Talend Support Portal for more direct assistance when you encounter problems with TDC environment construction, server configuration, etc.

Regards

Shong

Re: import metadata from talend bigdata to talend data catalog

mshafeeq — Mon, 13 Mar 2023 15:37:31 GMT

hello @shong (Talend Beijing Tech Co., Ltd.) ,

thanks a lot for support , i have issue when try to search on project path for example , i'm able to access DC machine only , how can i configure that i can access the other machines ?

Data catalog is installed on linux machine and Talend bigdata is installed on another one

same situation for HDFS , hive and impala , each one installed on different machines

can you please support me in this ?

thanks

Re: import metadata from talend bigdata to talend data catalog

kino77 — Tue, 06 Jun 2023 09:24:10 GMT

Hi @muhamd shafeeq,

I'm facing the same issue. If you want to access another machine and harvest from there you have to install an harvesting server and connect that to DC.

Apart from that, on the remote machine you have to clone your Git repositories in which your Talend Studio developed procedures was configured/saved. This will be the path that you have to specify in the bridge.