topic Re: Filter data in Talend Studio

Filter data

lmit — Tue, 29 Jan 2019 10:04:19 GMT

Hi All,

I have 2 files anf each file has data like below

File 1:

id|country|name

1|US|A

1|US|B

1|US|c

File 2:

id|country|name

1|US|A

1|US|B

1|UK|c

my output should be:

id|country|name

1|US|A

1|US|B

1|UK|c

1|US|c

If my Id's match and country column is different , my output should have both the records from file 1 and file 2

Can anyone suggest how to handle it

Thanks in advance,

lmit

Re: Filter data

Anonymous — Tue, 29 Jan 2019 10:27:32 GMT

Hi,

You can do as below.

a) Merge both data (since schema is same) using tUnite Component

b) Pass the output of tUnite to tUniqrow and select ID and country as key columns (columns to select unique rows).

c) Pass the unique component output to desired output for further processing.

Warm Regards,
Nikhil Thampi

Please appreciate our Talend community members by giving Kudos for sharing their time for your query. If your query is answered, please mark the topic as resolved 🙂

Re: Filter data

lmit — Tue, 29 Jan 2019 14:23:55 GMT

Thanks Nikhil Thampi for your reply

will try this method

Thanks,

lmit

Re: Filter data

Anonymous — Tue, 29 Jan 2019 14:26:56 GMT

Hi,

It should work 🙂

Please remember to mark the topic as resolved as it will help other Talend community members. Kudos are also welcome 🙂

Warm Regards,

Nikhil Thampi

Re: Filter data

lmit — Wed, 30 Jan 2019 05:26:19 GMT

Hi,

It worked but I missed one logic

When the country codes are matching the amount in the old file should append "-"sign

Its kind of CDC logic , I will be getting only one file then I need to lookup with the old file

Thanks,

lmit

Re: Filter data

Anonymous — Wed, 30 Jan 2019 05:53:29 GMT

Hi,

In that case, you need to use a different strategy. You need to use the new file as the main record flow and the old file as the lookup flow. Then you need to do the inner join between both files.

For all the records which have matching during old records, you can do the append sign. For all the records which do not have any match (which can be segregated by inner join reject as true option in tMap output flow) , you can pass to a different file.

Once this mechanism is done, you can unite both files and remove any duplicate value and do the downstream processing.

Warm Regards,
Nikhil Thampi

Please appreciate our Talend community members by giving Kudos for sharing their time for your query. If your query is answered, please mark the topic as resolved 🙂

Re: Filter data

Anonymous — Wed, 30 Jan 2019 09:29:25 GMT

@lmit

Could you please mark the topic as closed since your original query in the post has been resolved?

Warm Regards,
Nikhil Thampi

Please appreciate our Talend community members by giving Kudos for sharing their time for your query. If your query is answered, please mark the topic as resolved 🙂