Create multiple output files based on a unknown number or keys

[Anonymous]

I am trying to split one flow into multiple outputs based on a key. there are an unknown number of keys and every row with the same key is going to be outputted to a tFileOutputDelimited corresponding to that key. Any thoughts on how to go about doing this?

[Anonymous]

Hi Sabrina,
Thank you for your quick response
As I am unable to upload screen shots am providing my requirement:
Actually I have a huge CSV file. It has a date column. My data is from 2000 to 2016 year. 
All dates are in this format--"DD-MM-YYYY".
I would like to store each year data in a separate csv file. For example all 2016 data has to store automatically in separate csv file and 2015 data in a separate csv file in the below path 
(E drive/output/year/2016)
(E drive/output/year/2015)..............so on....
I hope you understand my requirement. It would be very helpful if you provide job design with screen shots. As i am very new to talend.
Kindly let me know if you have any concerns.
Thanks in advance.
Kind regards,
Asha

[_AnonymousUser]

Hi,
Can any one explain how to update records in  csv file?
My requirement is :
I have two csv file with same columns id, name.
(old.csv) id name
             1  Asha
             2  Jyothi
(New.csv) id name
               1 Jyothi
               2 Raj
               3 Vinay
So I need to update the new records for id 1 and 2 and insert 3rd record.
Please let me know if any concerns.
Kind Regards,
Asha

[_AnonymousUser]

I also have same scenario where i need to write data into files as per the data value of some column like 
(E drive/output/year/2016)
(E drive/output/year/2015)
where i am getting  2015 and 2016 in my data as a column value .
I am able to achieve this with standard job .
But I am facing the problem  doing this by  Bigdata Batch job (Spark ).
or suggest any other optimize way using spark batch job 

[Anonymous]

I also have same scenario where i need to write data into files as per the data value of some column like 
(E drive/output/year/2016)
(E drive/output/year/2015)
where i am getting  2015 and 2016 in my data as a column value .
I am able to achieve this with standard job .
But I am facing the problem  doing this by  Bigdata Batch job (Spark ).
or suggest any other optimize way using spark batch job 

I achived this by writing custom java code by saving RDD using multiple file format .
You can also use dataframes partition by save method 
Br 
Anuj

[premv258]

If you just want to split the input file based on row count. Then Use  tFileOutputDelimited.

Go to Advanced setting  and tick (Split output in several files) and you can mention number of row you need in each output file.