topic Re: Split to multiple files based on field in Talend Studio

Split to multiple files based on field

MattE — Fri, 15 Nov 2024 22:13:18 GMT

Hi,

I've got a redshift query which brings back some data, i want to split this data into multiple output csvs dynamically based on a field (company name). I know i could do this with the filter in tMap but that would require me to setup a filter per company beforehand and add / remove them each time which is no good. Is there another way to do this?

Any help would be appreciated.

Thanks

Re: Split to multiple files based on field

Anonymous — Fri, 06 Jan 2023 08:02:39 GMT

Iterate each record and set a dynamic output file, each output file only contains the data belong to the company name. eg:

tRedshitInput--main(row1)--tFlowIterate--iterate--tFixedFlowInput--main--tFileOutputDelimited

tFixedFlowInput: generate the current row.

set a dynamic output file path on tFileOutputDelimited:

"d:/file/"+ (String)globalMap.get("row1.companyNameColumn")+".csv"

Regards

Shong

Re: Split to multiple files based on field

MattE — Mon, 09 Jan 2023 12:02:10 GMT

Thanks @Shicong Hong this is perfect