topic Re: Create Dynamic Output Files Based on Input File in Talend Studio

Create Dynamic Output Files Based on Input File

Anonymous — Fri, 09 Mar 2018 03:49:00 GMT

Hi All,

I am new to Talend Open Studio 6.5 and exploring it for self-learning. I want to perform following task -

I have an input text file with region wise sales - sample data is as below -

Region, Sales

West, 400

West, 300

South, 500

South, 700

North, 200

East, 800

... and so on.

What I want to do is, create multiple text files from this source data. There should be 4 output text files created. West, South, North, and East (the flow should be dynamic and if there's another region added, it should create that new region name's file)

Each text file will only contain that particular region's data. Eg. West.txt will ONLY have west sales data, North.txt will ONLY have north sales data.

Can someone please help on this and guide me?

Thanks,

Sanket Kelkar.

Re: Create Dynamic Output Files Based on Input File

TRF — Sat, 10 Mar 2018 18:24:31 GMT

Hi,

As a trivial solution you may use a tMap with filter for output flows.

Here is how it looks like:

In this example I filter only for West and South regions but you just have to add output for North and East with the appropriate filters.

After the tMap, connect each output flow to the desired file, and that's all.

Hope this help.

Re: Create Dynamic Output Files Based on Input File

Anonymous — Mon, 12 Mar 2018 19:54:00 GMT

@Sanket , what is suggest by @TRF will work but the hardcoded condition.

What you are looking for is dynamic which this:

tfileinputdelim->tflowiterate->tFIxedflowinput (create 2 fields in out schema , declare the value of fileds as ctrl+space , values from tflowiterate variable)->tFIleoutdelim (in target path use lets say , "c:/myfiles/"+"_"+<region variable from flowtoiterate>+".txt" and also check 'append box'

try this proven method. any question ask.

if tomorrow u will add more region no problem , solution will work.

Re: Create Dynamic Output Files Based on Input File

Anonymous — Sat, 24 Mar 2018 04:54:19 GMT

Hi @snayak and @TRF,

Thank you for your inputs.

I have achieved the scenario as shown in the attached screenshot-

1. tFileInputDelimited - Gives me all the region names from the source file.

2. tUniqueRow - Gives me the list of unique regions.

3. tBufferOutput - Writes the list of unique regions in the buffer.

--- OnSubJobOk ---

4. tBufferInput - Retrieves the list of unique regions.

5. tFlowToIterate - Iterates through the list of unique regions and passes one region at a time to the next flow.

6. tFileInputDelimited - Accesses the entire source file to fetch actual data.

7. tFilterRow - Filters the rows only for the region for which the flow is iterating. Have put filter condition Region = ((String)globalMap.get("row3.Region"))

8. tFlowOutputDelimited - Creates a text file with the region name. The filename should be region name so have put "E:/Talend DI/Output/"+ ((String)globalMap.get("row3.Region")) + ".txt" as my filename.

The entire flow is dynamic and gives me what is expected.

Thanks,

Sanket Kelkar.

Regionwise_Dumps.png