topic Re: tUniqRow not working with tMap in Talend Studio

tUniqRow not working with tMap

zalibra — Thu, 23 Aug 2018 22:56:11 GMT

Hi all,

Created a test flow with tFileInputDelimited --> tUniqRow --> tFileOutputDelimited/tLogRow and everything works perfectly.

When I slightly modified the flow to obtain the information from the original source, tUniqRow does not seem to work. Here's my required flow:

tFileInputJSON --> tExtractJSONFields --> tMap --> tUniqRow --> tFileOutputDelimited/tLogRow. This doesn't seem to give me the distinct values.

My configuration of tFileInputJSON, tExtractJSONFields, tMap and tFileOutputDelimited/tLogRow are correct since the data is being parsed and mapped correctly. I just can't seem to do the distinct.

From the JSON, I parsed "baseAsset" and "quoteAsset" in two separate columns, which I then combined using tMap. From the tMap combined values, I need to do a distinct but it's not working.

For example, parsing the JSON gave me the values

EUR, PES, DIN in column A

USD, USD, EUR in column B

Using tMap, I put all values in column A - EUR, PES, DIN, USD, USD, EUR

Now I need to do a distinct and get the values EUR, PES, DIN, USD

I'm attaching the screenshots of tExtractJSONFields, tMap and tUniqRow. Any help would be greatly appreciated.

Re: tUniqRow not working with tMap

TRF — Thu, 23 Aug 2018 23:05:40 GMT

When you do colA + "\n" + colB in tMap, you don't receive 2 rows in tUniq, you receive 1 row with a linefeed in the middle.
Try to add a tFileOutputDelimited after the tMap. Then add a subjob starting with a tFileInputDelimited + tUniqRow + what you want. This subjob must be connected to the previous one by a onSubjobOk trigger. As rows are written to a file, when you read the file again you will get 2 rows as expected.

Re: tUniqRow not working with tMap

zalibra — Fri, 24 Aug 2018 00:29:08 GMT

Thank you, TRF, for getting back to me. I think I've found another solution.

In tMap, instead of combining the columns using "\n", I used "Join table output2 linked with output1". Now, if I go through the flow tExtractJSONFields --> tMap --> tUniqueRow --> any output, it gives the desired results.

tMap join outputs.jpg