topic Re: Split the Table into multiple Files in Talend Studio

Split the Table into multiple Files

McJingles — Wed, 20 Nov 2019 07:43:28 GMT

Hi! In the table, there are 16,000 records in it. In the Year column, it has the values from 1960-2019. I want to split the data into multiple files by the Year basis. Any suggestions? Thanks in advance.

Re: Split the Table into multiple Files

Anonymous — Wed, 20 Nov 2019 08:04:12 GMT

Hi,

You can use tMap component for this. The input will be your table data and you can have multiple output rows depending on your requirement of the values. For this you can use, expression filter available in tMap to filter out the year ranges according to your requirements. Each "output" row can then be connected to different file.

Here is a sample example of what I'm trying to say..

Hope this helps.

Don't forget to give kudos if this helps and accept it as solution if it works as per your needs.

Regards,

Bhagwat

Re: Split the Table into multiple Files

McJingles — Wed, 20 Nov 2019 08:13:13 GMT

Thanks. But the year is 1960-2019. I have to insert nearly 50 output delimited components to split the file. Is there any other option to simplify this?

Re: Split the Table into multiple Files

manodwhb — Wed, 20 Nov 2019 08:45:10 GMT

@McJingles , Please check the below way.

1) read the unique years from table or file and iterate those using tFlowtoIterate as shown in below.

Re: Split the Table into multiple Files

MaheswarReddy — Wed, 20 Nov 2019 08:48:55 GMT

Hi @McJingles ,

Please find out the distinct years available in the table and iterate the years one after the other as the filter in the tDBinput component and just iterate the flow .

It will be like this,

step1:-

tDBinput component -- tfileoutput (Find the distinct years and store in tfileoutput)

step2:-

tfileinput--> Flow to iterate --> tDBinputcomponent(use filter condition and pass the year value from iterate component) -->tMap -- tfileoutput(in this use the year name as filename parameter and pass the same year value from iterate component, so that multiple files are generated)

This is very simple process than creating so many outputs in tmap and you end up creating 100 outputs if you have 100 distinct years .

Please try the two steps which i mentioned.

Let me know if you need any help.

Thanks,

Maheswar Reddy.

Give Kudos if you like it , and accept as solution if it suits your question.

Re: Split the Table into multiple Files

McJingles — Wed, 20 Nov 2019 09:12:29 GMT

Thanks @manodwhb But, Can you please show me the component section?

What is in the input delimited 2 component?

And I don't know how to use the flow to iterate. can you briefly explain this process?

Re: Split the Table into multiple Files

manodwhb — Wed, 20 Nov 2019 09:25:05 GMT

@McJingles ,check the below link to know more about tFlowToIterate .

https://help.talend.com/reader/wDRBNUuxk629sNcI0dNYaA/WqJCkTE5~5DDaSFvtNXZnQ