topic Re: File Data Type: Dynamic - Execute Unix Commands through tSystem and compare the output to create a log file in Talend Studio

File Data Type: Dynamic - Execute Unix Commands through tSystem and compare the output to create a log file

jaink — Sat, 16 Nov 2024 07:57:25 GMT

Hi,

I need to validate whether the header of file is expected or not. The expected header is passed as a context variable.

As i am using dynamic data type while reading the file, i cannot process it by any other means.

Input File - a.txt
A,B,C
1,2,3
4,5,6,7
8,9

Below commands were used, and output of tsystem is stored in global variables. But when the global variables are compared in tmap, log file is always created irrespective of whether the header matches or not.

tSystem1 -->(main) tSystem2 -->(main) tMap -->(main) tOutputFileDelimited

tSystem1 - "/bin/bash -c echo " + context.Header
tSystem2 - "/bin/bash -c echo `head -1 " + context.InputFile + "`"
tMap - ((String)globalMap.get("tSystem_1_OUTPUT")).equals(((String)globalMap.get("tSystem_2_OUTPUT"))) ? 0 : 1
Link Condition - var == 1
Output Message - Header not valid
tOutputFileDelimited - Create a log file

But, somehow, even when the header is as expected, the log file is created with output message.
What am i missing here? Also, if there is any other way to achieve the same with dynamic data type of the file, please let me know!

Re: File Data Type: Dynamic - Execute Unix Commands through tSystem and compare the output to create a log file

Anonymous — Tue, 17 Jul 2018 06:45:40 GMT

Hi
The global variables should be used after tSystem execution. Change the job to:
tSystem1--oncomponentok--tsystem2--oncomponentok--tFixeFlowInput--main--tMap-->tfileOutputDelimited

on tFixedInputFlow: output the value of global variable, define two columns with string type.
c1 (String)globalMap.get("tSystem_1_OUTPUT"))
c2: ((String)globalMap.get("tSystem_2_OUTPUT"))

Regards
Shong

Re: File Data Type: Dynamic - Execute Unix Commands through tSystem and compare the output to create a log file

jaink — Tue, 17 Jul 2018 06:59:32 GMT

@shong

Even with this approach, the file is always created irrespective whether the header is same or not!

Re: File Data Type: Dynamic - Execute Unix Commands through tSystem and compare the output to create a log file

Anonymous — Tue, 17 Jul 2018 07:46:23 GMT

Hi
I don't understand the commends you are using to validate the header, what's the output of global variables? Try to print the value on a tJava to see if they are expected values.
tSystem1--oncomponentok--tsystem2--oncomponentok--tjava

Shong