tFileOutputParquet - creating Parquet files

[RMotta2408]

Hi there,

I created this very simple Job:

There's no problem when I run it in Talend Cloud Big Data.

But when I run the same Job on Talend Cloud Data Fabric, on the same computer, it gives me the followinh error:

"Starting job ReadParquetFile at 12:41 09/05/2023.

Error: Could not find or load main class Dhadoop.home.dir=C:\hadoop-3.3.0\bin

Caused by: java.lang.ClassNotFoundException: Dhadoop.home.dir=C:\hadoop-3.3.0\bin

Job ReadParquetFile ended at 12:41 09/05/2023. [Exit code = 1]".

Can anyone please help me solve this issue?

I appreciate it.

Rui

[Anonymous · Accepted solution]

Hello @Rui Motta​ ,

It's a well known issue about hadoop on windows

see https://cwiki.apache.org/confluence/display/HADOOP2/WindowsProblems

The solution is:

1. git clone the repository https://github.com/steveloughran/winutils to your local machine
2. Set the environment variable %HADOOP_HOME% to point to the directory above the BIN dir containing WINUTILS.EXE, it need to restart the studio make it take into account.

Or

setup a jvm parameter -Dhadoop.home.dir in the job's Run->Advanced settings page as the below

[RMotta2408]

@Aiming Chen​ , thank you so much for your help.

It worked like it was supposed to.

I really appreciate all the help you've been giving.

Rui