Big Data Spark Job - Process job based on file name

[Anonymous]

Hi,

I am creating big data spark job and have to check file name before proceeding to the specif workflow like

checkfilename -> (if contains "abc") -> then read file / process file

      |

(if filename contains "xyz") -> then read file/ diff process

OR

tInputfileDelimited -> Checkfilename (if contains "abc") -> then process file

                                     |

                                 (if filename contains "xyz") -> then diff process

Which component I have to use to implement above scenario. I tried to use tJavarow but runif has several limitation. We can connect tjavarow ->(runif)-> tJava only.

[Anonymous]

Hi,

In a standard job, here is a component tfileproperties which can create a single row flow that displays the properties of the processed file.

Best regards

Sabrina

[mamohan]

I have similar requirement and tfileproperties is not available in spark job. Can you please suggest for a spark job.