merge some csv files in one (merge the different columns)

[Anonymous]

Hello!!

I have a requirement a bit unusual to do in Talend.

I need to merge some csv in the next way: adding columns to complete the schema

I'll put an example to be able to explain it:

File 1:

(header): var1 | var2 | var3

row 1:      abc   |        | 123

row 2:      sdd   | true | 234

File 2: 

(header): var4 | var5 

row 1:      abc   | 11-02-2019       

row 2:      sdd   | 21-12-2019          

row 3:      sdd   | 28-07-2019 

File 3:

(header): var6 | var7 | var8

row 1:      true   | 1     | +34

row 2:      false  | 0     | +44

MERGE FILE:

(header):  var1 | var2 | var3 | var4 | var5            |var6  | var7 | var8 |

row 1:       abc   |        | 123  | abc  | 11-02-2019 | true  |   1    | +34  |                 

row 2:       sdd   | true | 234  | sdd  | 21-12-2019 | false|   0    | +44  |

row 3:                |        |         | sdd  | 28-07-2019 |         |         |         |      

I really appreciate your help!!

Thanks in advance,

Jorge 

[Anonymous · Accepted solution]

Hi Jorge,

      The most easy way is to add a sequence number within a tMap after reading each file so that you will get a key to join between files. 

       But you do not have to use Inner join so that you can get all possible results. Then use a second tMap to join between these flows based on sequence number.

       This should do the magic. If you are facing any difficulty, please let us know but I would recommend to try it yourself for sometime so that you will get more idea about how data merging happens within Talend Studio.

Warm Regards,
Nikhil Thampi

Please appreciate our Talend community members by giving Kudos for sharing their time for your query. If your query is answered, please mark the topic as resolved 🙂

[akumar2301]

Agree with Nikhil 

for line like 

row 3:                |        |         | sdd  | 28-07-2019 |         |         |         |      

make sure sequence number is added not only to exisitng records but also to emptry records till the maximum count of record reach.

e;G;

File 1:

id| var1 | var2 | var3

1|abc   |        | 123

2| sdd   | true | 234

3|

File 2: 

IDvar4 | var5 

1|abc   | 11-02-2019       

2|sdd   | 21-12-2019          

3| sdd   | 28-07-2019 

File 3:

ID|var6 | var7 | var8

1| true   | 1     | +34

2| false  | 0     | +44

3|

[Anonymous]

Thanks you Nikhil!
It worked perfectly!!

Best regards,
Jorge