topic Re: Create floders based on column values in Talend Studio

Create floders based on column values

Anonymous — Tue, 01 Aug 2017 08:09:06 GMT

Hi Team,

I have below scenario.

Source : Oracle/File

Target: File/Oracle

Source Columns

-------------------------

snlno, txn_dt

1, 2017-01-07

2,2017-01-06

1,2017-01-05

2,2017-01-05

Required output:

Directory:

d:/output/6jan2017/6thdayrecords.csv

d:/output/5jan2017/5thdayrecords.csv

d:/output/7jan2017/7thdayrecords.csv

I want store all distinct dates in context variable and use to create folder based on dates, store records inside directory only specific to that date.

Thanks

Shridhar

Re: Create floders based on column values

Anonymous — Tue, 01 Aug 2017 09:35:04 GMT

There are a couple of things to think about here. First of all, you will need to be able to use a bit of Java to manipulate your date information to get the required String to build your folder path. Take a look here: https://www.talendforge.org/forum/viewtopic.php?id=17451

The above should give you an idea of how to achieve this using Talend system routines.

The next bit to consider is a little bit more complicated. You can create your folder when you create a file that needs to be written in that folder. That is easy enough. SO you will not have to worry about preemptively creating your folder and then writing your file there. But it looks like you will need to keep changing the file you are writing to in a single flow.....which does not work out of the box. You will need to group your data by output file and iterate over that group so that your file is created and written to in separate flows......which is a pain.

Or (if you are not scared of a little Java) you can follow what I have done in this tutorial....https://www.rilhia.com/tutorials/load-data-dynamic-number-files

Re: Create floders based on column values

Anonymous — Wed, 02 Aug 2017 06:46:58 GMT

Hi,

Thanks for valuable input.

I am not good in java, Let me go though as suggested inputs.

Thanks

Shridhar