Split to multiple files based on field

[MattE]

Hi,

I've got a redshift query which brings back some data, i want to split this data into multiple output csvs dynamically based on a field (company name). I know i could do this with the filter in tMap but that would require me to setup a filter per company beforehand and add / remove them each time which is no good. Is there another way to do this?

Any help would be appreciated.

Thanks

[Anonymous · Accepted solution]

Hi

Iterate each record and set a dynamic output file, each output file only contains the data belong to the company name. eg:

tRedshitInput--main(row1)--tFlowIterate--iterate--tFixedFlowInput--main--tFileOutputDelimited

tFixedFlowInput: generate the current row.

set a dynamic output file path on tFileOutputDelimited:

"d:/file/"+ (String)globalMap.get("row1.companyNameColumn")+".csv"

Regards

Shong

[MattE]

Thanks @Shicong Hong​ this is perfect