Eliminate matching data from the file

Karuetl — Fri, 07 Dec 2018 16:59:32 GMT

I have below source file and data from table A , if file contains any of the values from Table A then eliminate it from the file .. need help in either Java or anything

One logic i thought is convert entire row from file as string into Java so that i can compare and mark true or false and then filter , but need help to write java code

if any better logic please let me know

Source file as below (Dynamic columns positions are not known)

-----------------------------

7465721^2018-10-22^^^34^2018-10-22 00:59:01^2018-10-22 ^ln-01@abc.com^663986903^Auth
3486223^2018-10-22^^^34^2018-10-22 00:59:05^2018-10-22 ^ln-01@euf.com^708548590^Auth
8975432^2018-10-22^^18459068^607^2018-10-22 00:59:12^2018-10-22 ^mmcvetril^185245600^OnlineAcc

9642532^2018-10-22^^84441111^607^2018-10-22 00:59:12^2018-10-22 ^acboid^346543245^webv

Table A i have below values

708548590

965354999

346543245

So I want the output as

7465721^2018-10-22^^^34^2018-10-22 00:59:01^2018-10-22 ^ln-01@abc.com^663986903^Auth
8975432^2018-10-22^^18459068^607^2018-10-22 00:59:12^2018-10-22 ^mmcvetril^185245600^OnlineAcc

Re: Eliminate matching data from the file

TRF — Sun, 09 Dec 2018 12:53:32 GMT

This is a very common and simple use case.

Use a tFileInputDelimited to read the input file with "^" as a field separator.

Then with a tMap join the flow with the lookup table to exclude undesired lines ("inner join" + catch "inner join reject" options).

Re: Eliminate matching data from the file

Karuetl — Mon, 10 Dec 2018 14:37:10 GMT

On what basis the joins be mentioned , as we dont know the position of this column has the value

topic Re: Eliminate matching data from the file in Talend Studio

Eliminate matching data from the file

Re: Eliminate matching data from the file

Re: Eliminate matching data from the file