Hadoop client setup

Anonymous — Sat, 16 Nov 2024 11:01:03 GMT

Hello!
My cluster is on Linux and our ETL infrasturcture is on Windows. What I am coming up against is that on Windows for Talend BigData edition to run, there needs to be a local hadoop client installed. While there is no good documentation available, does anyone have steps on how to get going with setting up a Hadoop client on Windows and configuring it to connect to the cluster on Linux ? Also the same configs that needs to be keyed in Talend itself (tHDFS). I tried a few options, but does not seem to work.
_Ben

Re: Hadoop client setup

Anonymous — Thu, 01 Oct 2015 23:16:18 GMT

Hi Ben,
I'm using both Hive and HDFS components with CDH 5.4 on windows 7 in GUI mode and deploying in standalone to Solaris.
Neither Windows nor Solaris have any hadoop client stuff installed and they both work fine (except for an exception on windows about an exe that is not a blocker).
For hdfs I am using the Cloudera and CDH 5.4(Yarn). For Hive I am using the same in Stand Alone Mode.
If your circumstances are different and you really need the client installed it's possible. One of our admins here did it to help testing kerberos, but I have not done it myself. He used these two links to get it done.
wiki.apache.org/hadoop/Hadoop2OnWindows
www.barik.net/archive/2015/01/19/172716/

Hope that helps.
Pete

topic Hadoop client setup in Talend Studio

Hadoop client setup

Re: Hadoop client setup