topic Re: merge data from multiple files into one in Talend Studio

merge data from multiple files into one

Anonymous — Mon, 23 May 2016 13:44:46 GMT

Hi,
I have plenty of spreadsheets with same columns but different data (the old database extracted the data grouped in small fractions by separate files). I need to merge all those files into one only. Is there an easy way to do this? Many thanks for help

Re: merge data from multiple files into one

Rajesh_Reddy1 — Mon, 23 May 2016 14:18:52 GMT

If you have same columns in all the files then Use tFileList component.
https://help.talend.com/search/all?query=tFileList&content-lang=en

Re: merge data from multiple files into one

Anonymous — Fri, 27 May 2016 13:31:58 GMT

Hi,
I thought it was working as suggested but I have actually just noticed that the tFileList component iterates through 12 folders where I have my files, but the output is taken from the very last folder only. What am I doing wrong? See attached screenshots to see the component settings, all files in subdirectories are named c5_contact. Many thanks for help.

Re: merge data from multiple files into one

Anonymous — Fri, 27 May 2016 13:42:39 GMT

You're iterating files, so make sure you append to your spreadsheet - "Append existing file"
Check out the other options to make sure you get what you're after.

Re: merge data from multiple files into one

Anonymous — Fri, 27 May 2016 14:54:45 GMT

Good point thanks but if I use 'Apend Existing File', the output is same.. and if I add 'Apend Existing Sheet' (dont really know what the difference is) I get the error attached.. Im not code oriented so I dont understand it.

Re: merge data from multiple files into one

Anonymous — Sat, 28 May 2016 06:16:26 GMT

How many rows are you attempting to put in this spreadsheet?

Re: merge data from multiple files into one

Anonymous — Mon, 30 May 2016 07:59:35 GMT

In this particular example I was trying with 12 files having around 20 000 rows in total, but Im going to actually need 2x more later. Thanks