topic Filter output rows in Talend Studio

Filter output rows

stvn — Wed, 13 Feb 2019 12:17:59 GMT

Hello Talend Community,

I'd like to have your help on an issue that I face. the following job have a path with an input (/Default/Tools/Articles/Documents/Documents_Extract/) and as an ouput these result:
/Default/
/Default/Tools/
/Default/Tools/Articles/
/Default/Tools/Articles/Documents/
/Default/Tools/Articles/Documents/Documents_Extract/

Now I'd like to filter those output rows in order to have unified values such as:

/Default/

/Tools/
/Articles/
/Documents/
/Documents_Extract/

Could you help me whit the appropriate components to use? I'don't know which component should I use.

Thank you in advance

Steve

Re: Filter output rows

fdenis — Wed, 13 Feb 2019 12:30:00 GMT

you can try to use this on a tMap
row1.filepath --> row1.filepath.replaceAll("(/[^/]+/)$", "$1")

Re: Filter output rows

stvn — Thu, 14 Feb 2019 07:12:57 GMT

Hi François,

thank you for your reply.

My first question is: What does "$1" represent?

if there is a second row start by : /Default/Tools/Articles/Files/FilesAfterExtract/ Is the tMap is goind to give me as an output these result?:

/Default/Tools/Articles/Documents/Documents_Extract/

/Default/Tools/Articles/Files/FilesAfterExtract/

/Default/
/Tools/
Articles/
/Documents/
/Documents_Extract/

/Files/

/FileAfterExtract/

I want to keep variable that has not been created yet.

thank you for your time and your help

Re: Filter output rows

fdenis — Fri, 15 Feb 2019 10:29:04 GMT

you can get help on regula expression with java on multiple site.
but on our case $1 represent the value inside parenteses(/[^/]+/)
/ is /
[^\]+ is something without \ one or more time.
$ is the end of the line

row1.filepath.replaceAll("(/[^/]+/)$", "$1")
just give you the last element between / and /

Regards,

Re: Filter output rows

stvn — Mon, 18 Feb 2019 14:31:56 GMT

Hi François,

Thank for your help, I tried your recommended regex and unfortunately it did not removes any duplicate Values.

You will find below the result:

My idea was to have at the end

/Default/
/Tools/
/Articles/
/Documents/
/DocFirst/

If there is a second line started by /Default/Tools/Articles/DocumentsFirst/DocumentsSecond then the output would be:

/DocumentsFirst/

/DocumentsSecond/

could you advise me any other suggestion? Thank for your help.

Warm Regards,

Steeve

Re: Filter output rows

manodwhb — Tue, 19 Feb 2019 05:18:30 GMT

@stvn, Please check the below solution for your case.

Re: Filter output rows

stvn — Tue, 19 Feb 2019 21:10:18 GMT

@manodwhb

Hi I'd like to thank you for the provided solution. It works pretty well but the think is, I did not express very well my needs. I apologize for that. I have a csv file as input:

/Default/Tools/Documents/Articles/Factures/
/Default/Tools/Documents/Articles/Factures/Factures_Comptable/
/Default/Tools/Documents/Articles/Factures/Factures_Comptable/Année

And I'd like as an ouput:

/Default/
/Default/Tools/
/Default/Tools/Documents/
/Default/Tools/Documents/Articles/Factures/

/Factures_Comptable/

/Année

After the first line, I don't have any other duplicates variable. I implement the job below in order to filter my rows and keep only the data as shown in the example above. Could you recommend me any other solutions?

Thank you for your time and consideration.

Warm regards,

Stevn