Unlock a world of possibilities! Login now and discover the exclusive benefits awaiting you.
Hi,
I am creating big data spark job and have to check file name before proceeding to the specif workflow like
checkfilename -> (if contains "abc") -> then read file / process file
|
(if filename contains "xyz") -> then read file/ diff process
OR
tInputfileDelimited -> Checkfilename (if contains "abc") -> then process file
|
(if filename contains "xyz") -> then diff process
Which component I have to use to implement above scenario. I tried to use tJavarow but runif has several limitation. We can connect tjavarow ->(runif)-> tJava only.
Hi,
In a standard job, here is a component tfileproperties which can create a single row flow that displays the properties of the processed file.
Best regards
Sabrina
I have similar requirement and tfileproperties is not available in spark job. Can you please suggest for a spark job.