topic Re: How do i combine rows from same data source after they are splitup? in Talend Studio

How do i combine rows from same data source after they are splitup?

Anonymous — Tue, 18 Apr 2017 21:24:12 GMT

I want to combine output of tMap2,tMap4,tMap5 , so that I can write the final output in 1 file. Each of tMap components has comumns A,B,C. I want to combine them by tMap2.A=tMap4.A=tMap5.A && tMap2.B=tMap4.B=tMap5.B and want to keep C of each of the three tMap components.

Any help on how to do this , much appreciated..!

Re: How do i combine rows from same data source after they are splitup?

vapukov — Tue, 18 Apr 2017 22:46:19 GMT

direct - it is not allowed, but You can store result of each tMap into memory, than join all 3

You can use for this tHashInput / tHashOutput component

in default palette both not present, You need open menu File -> Project Settings -> Designer -> Palette settings and enable them for Your project.

Re: How do i combine rows from same data source after they are splitup?

Anonymous — Wed, 19 Apr 2017 20:15:51 GMT

well in Op's case it is possible to simply add the data in the same file in separate flow while keeping the file in append mode and disabling the headers

Re: How do i combine rows from same data source after they are splitup?

cterenzi — Wed, 19 Apr 2017 21:57:49 GMT

It sounds like the OP wants output along the lines of A, B, C1, C2, C3 rather than just writing all output to one place. In that case, Vapukov's solution would be appropriate. Really any intermediate data store will work-- temp files or database tables being other alternatives. HashMaps are the simplest and fastest option, I think.

Re: How do i combine rows from same data source after they are splitup?

Anonymous — Thu, 20 Apr 2017 21:00:37 GMT

ok, I have removed TFiledelimetdout and tried above mentioned solutions. Could anyone tell me how the data is flowing through this workflow. Like I have used 3 tHashOutut -> 1 tHashInput and wrote result in delimeted File. Bu the final result has duplicates. A record that is filtered through the First line of Job design, goes is repeated 7 times.
So, I wonder how exactly data is lowing across this?

Besides, what I really need is join condition but the above mentioned solution is working on Append strategy

Re: How do i combine rows from same data source after they are splitup?

vapukov — Thu, 20 Apr 2017 21:08:36 GMT

can't understand without picture, but look different with Your original idea

look for the Hashoutput same as for tLogRow or tFileOutpitDelimited

You can do JOIN and etc - this changes not add duplicates by it self if not present in logic

Re: How do i combine rows from same data source after they are splitup?

Anonymous — Thu, 20 Apr 2017 21:17:09 GMT

This is what I am doing: But this strategy is appending the data of tHashInput1 and tHashInput2. What I actually need is to Join the data on 1 of the 5 coloumns of data sets stored in tHashInput1, tHashInput2.?

Re: How do i combine rows from same data source after they are splitup?

vapukov — Thu, 20 Apr 2017 22:15:42 GMT

the proper answer always depends from - what You do and try to achieve, what data, what in Your filters, what in tMaps, what JOIN conditions?

duplicates not come from nowhere if You do not have them original or not create them by Your self on previous steps

if we return to original question - You save information in 3 separate and ask - how to JOIN all 3 on same Job

tHashOutput1/2/3 -> tHashInput1/2/3 is answer for this question, same for 300% as You do with csv files and tLogRow

is it have duplicates? - I do not know, it depend from whole Job logic