topic Re: Talend Data Masking component throwing error when used in Spark job in Talend Studio

Talend Data Masking component throwing error when used in Spark job

Anonymous — Sat, 16 Nov 2024 10:00:57 GMT

org.talend.dataquality.datamasking.Function cannot be resolved to a type, this is the error thrown when used with tJava component.
We are trying to read data from a HDFS file by tFileInputFullRow and then we are doing some computations and when I send a RDD<String> to the tDataMasking component, error is thrown.
We tried with the simple row generator connected to tDataMasking in Spark job, it throws same Error.
(Same code is working for Standard job).
We have checked the plugins and Spark job and Spark streaming is supported for tDataMasking component .
I am using Talend 6.2.1
Regards,
Deepan

Re: Talend Data Masking component throwing error when used in Spark job

Anonymous — Thu, 09 Mar 2017 06:45:53 GMT

Hi Deepan,
Would you mind posting your current job setting screenshots into forum which will be helpful for us to address your issue?
Best regards
Sabrina

Re: Talend Data Masking component throwing error when used in Spark job

Anonymous — Thu, 09 Mar 2017 10:36:47 GMT

The first screenshot is when I use the row generator for a Spark job.

The second screenshot is when I am using it for a custom Spark job.

Regards,
Deepan