topic Create multiple Output files based on Input Data in Talend Studio

Create multiple Output files based on Input Data

rajesh90 — Fri, 15 Nov 2024 23:43:47 GMT

Hello All,

I'm looking to write multiple output files based on the input data.

The sample data in my input csv file looks something like below,

NEWYORK,1

NEWYORK,2

BERLIN,1

BERLIN,2

BERLIN,3

TOKYO,1

GERMANY,1

GERMANY,2

GERMANY,3

GERMANY,4

What I want is to create multiple output files like below,

NEWYORK.csv

BERLIN.csv

TOKYO.csv

GERMNANY.csv

The above files should only contain the data of the respective countries. Ex. NEWYORK file only contains, NEWYORK data.

The number of countries can keep changing.

Can you help to provide a solution for this.

Thank you.

Re: Create multiple Output files based on Input Data

Anonymous — Fri, 17 Sep 2021 03:28:46 GMT

You need to iterate each row and set a dynamic output file name. eg:

tFileInputDelimited--row1--tFlowToiterate--tFixedFlowInput--main--tFileOutputDelimited

on tFixedFlowInput: generate the current record.

on tFileOutputDelimited, set a dynamic output file name:

"D:/file/"+(String)globalMap.get("row1.columnName")+".csv"

note: columnName is column name of the first column defined on tFileInputDelimited

Regards

Shong

Re: Create multiple Output files based on Input Data

Mahamoutou — Fri, 17 Sep 2021 10:03:21 GMT

You could these below steps to get job done.

1-- a whole job

2--input file

3--tUniqRow to get unique value of the city column

4--tFlowToIterate to make Talend iterate through those unique city values

5--Again input file

6--tFilterRow to filter out rows based on the current iteration value ( You get the current iteration city value from the Outline view >> TFlowToIterate ((String)globalMap.get("city_var")) ).

7--output files, make sure you put again this variable + ((String)globalMap.get("city_var")) +".csv" after your directory path

and you are done.