Split the Table into multiple Files

[McJingles]

Hi! In the table, there are 16,000 records in it. In the Year column, it has the values from 1960-2019. I want to split the data into multiple files by the Year basis. Any suggestions? Thanks in advance.

[Anonymous]

Hi,

You can use tMap component for this. The input will be your table data and you can have multiple output rows depending on your requirement of the values. For this you can use, expression filter available in tMap to filter out the year ranges according to your requirements. Each "output" row can then be connected to different file.

Here is a sample example of what I'm trying to say..

Hope this helps.

Don't forget to give kudos if this helps and accept it as solution if it works as per your needs.

Regards,

Bhagwat

[McJingles]

Thanks. But the year is 1960-2019. I have to insert nearly 50 output delimited components to split the file. Is there any other option to simplify this?

[manodwhb]

@McJingles , Please check the below way.

1) read the unique years from table or file and iterate those using tFlowtoIterate  as shown in below.

[MaheswarReddy]

Hi @McJingles ,

Please find out the distinct years available in the table and iterate the years one after the other as the filter in the tDBinput component and just iterate the flow .

It will be like this, 

step1:-

tDBinput component -- tfileoutput (Find the distinct years and store in tfileoutput) 

step2:-

tfileinput--> Flow to iterate --> tDBinputcomponent(use filter condition and pass the year value from iterate component) -->tMap -- tfileoutput(in this use the year name as filename parameter and pass the same year value from iterate component, so that multiple files are generated)

This is very simple process than creating so many outputs in tmap and you end up creating 100 outputs if you have 100 distinct years .

Please try the two steps which i mentioned.

Let me know if you need any help.

Thanks,

Maheswar Reddy.

Give Kudos if you like it , and accept as solution if it suits your question.  

[McJingles]

Thanks @manodwhb But, Can you please show me the component section?

What is in the input delimited 2 component?

And I don't know how to use the flow to iterate. can you briefly explain this process?

[manodwhb]

@McJingles ,check the below link to know more about tFlowToIterate .

https://help.talend.com/reader/wDRBNUuxk629sNcI0dNYaA/WqJCkTE5~5DDaSFvtNXZnQ