topic Re: Splitting data to separate files, dynamically in Talend Studio

Splitting data to separate files, dynamically

Gadik — Tue, 08 Jul 2014 16:50:59 GMT

Hi!
I have a data in table that need to be exported into different files, based on value in "category" field.
The number of distinct values may be dynamic (e.g. cannot use tMap component and hard-code the values).
File name will include the category string.
Columns:
ID | VALUE1 | VALUE2 | CATEGORY
1 abc abc A
2 abc xyz B
3 abc abc A
According to example above, 2 files should be created , e.d. /data/file_A.csv, /data/file_B.csv
Any suggestions?
Thanks!

Re: Splitting data to separate files, dynamically

Anonymous — Wed, 09 Jul 2014 04:27:16 GMT

Hi gadib,
Is this related forum working well for your use case?
https://community.talend.com/t5/Archive/Split-a-input-file-into-multiple-output-files/td-p/189235

Re: Splitting data to separate files, dynamically

Anonymous — Wed, 09 Jul 2014 09:01:44 GMT

Hi Gabib,
Use tfileinput-->tMap--(Get two output by applying a filter condition for your records)
This would help
Thanks
Vaibhav

Re: Splitting data to separate files, dynamically

Gadik — Wed, 09 Jul 2014 09:09:51 GMT

Thank you.
The link by xdshi is useful, however it implies that the large table with data will be selected several times, for each distinct value in "category" field. Which is very not effective.