topic Re: tMD5Sum component in Talend Studio

tMD5Sum component

Anonymous — Wed, 01 Aug 2018 20:11:11 GMT

Hi,

I had one scenario in which I need to compare the data with in the file.

for example: file1 has data:

This is book.

This is car.

This is book.

In this I need to know how many lines are duplicate.

Thanks in advance..!!

Re: tMD5Sum component

vboppudi — Wed, 01 Aug 2018 21:02:01 GMT

Please specify expected output.

Regards,

Re: tMD5Sum component

aashish_21nov — Thu, 02 Aug 2018 06:32:36 GMT

you can do so by tAggregateRow component by follow below flow:

tfileinput*--> tAggregaterow-->tfileoutput*

In tAggregaterow:

*group by data/or whatever be a name of column

*operation count(data/or whatever be a name of column)

Re: tMD5Sum component

Anonymous — Thu, 02 Aug 2018 14:17:49 GMT

Hi,

The expected o/p will be:

This is book. was come 2 times in a file

Regards,

Akash Rastogi

Re: tMD5Sum component

David_Beaty — Thu, 02 Aug 2018 14:21:28 GMT

I think the component you're after is tUniqRow

Re: tMD5Sum component

Anonymous — Thu, 02 Aug 2018 18:40:56 GMT

Hi David,

I think tUniqRow component is used to filtering the unique and duplicate records.

But in my scenario I want the exact count how many times that particular sentence had come.

-Akash

Re: tMD5Sum component

Anonymous — Thu, 02 Aug 2018 18:45:34 GMT

Hi Aashish,

Can you please more elaborate. I am unable to understand the solution.

My data is in file how can I use tAggregateRow on that.

-Akash

Re: tMD5Sum component

aashish_21nov — Fri, 03 Aug 2018 07:44:26 GMT

@rastogi_akashfor reading a file use tfileinputdelimited and let say your file having schema like sentence string type then use tmap and generate output with same sentence column two times,

So now we have two output column first sentence and second column (named as cnt) then add tggregaterow edit schema accordingly change datatype of cnt from string to integer and add sentence to group by and cnt to operation with function type count

your output will be :

sentence;cnt

a;1

b;2

c;1

you can use this result as per your requirement.