topic Re: Split into multiple files based on Date Value from Source in Talend Studio

Split into multiple files based on Date Value from Source

Karuetl — Fri, 11 Jan 2019 17:44:32 GMT

need help to achieve that We are source records with different dates and want to split into different files based on the month of those dates

Example

I have input data from source table as below

Col1 Col2 Col3
--------------------------
abc newyork  2018-10-15
abc newjersey 2018-10-12
abc newyork   2018-11-15
abc newyork   2018-12-06

Output should have 3 files

one for Month 201810

abc newyork  2018-10-15
abc newjersey 2018-10-12

one for Month 201811

abc newyork   2018-11-15

one for Month 201812

abc newyork   2018-12-06

Re: Split into multiple files based on Date Value from Source

akumar2301 — Fri, 11 Jan 2019 18:04:39 GMT

read file --> tjavarow ( set global var like filenameYYYYmm with yyyymm of last col --> output file ( filename should be globalvar (filenameYYYYmm ) append mode.

this should work

Re: Split into multiple files based on Date Value from Source

vapukov — Mon, 14 Jan 2019 03:13:10 GMT

I afraid - this is will not work

filename defined before subjob start

need insert tFlowToIterate which could define and filename and store row into memory (hash or variables) and 2nd subjob - read data and append them to file

Re: Split into multiple files based on Date Value from Source

akumar2301 — Mon, 14 Jan 2019 10:06:29 GMT

Thanks @Vapulkov ,

you are right . I did not thought about it.

I was ablr to do it with below steps

Read file -- > add col4 which in YYYYMM format of Col3(TjavaRow) --> file uniq on col4 in ( tUniqRow) -- > tflowtoItrate( put col4 in GlobalVar) --> read file again --> tFilterRow on GlobalVar --> tFilerOutPut ( Filename based on globalVar)

we could use tHashINput/output if we do not want to read files multiple time.

Re: Split into multiple files based on Date Value from Source

akumar2301 — Mon, 14 Jan 2019 10:08:14 GMT