topic Re: delimiter output file with increasing file name in Talend Studio

delimiter output file with increasing file name

pawe84 — Sat, 16 Nov 2024 05:31:49 GMT

In this small example there is just a query which will export the values into a csv.

The output file is called "List0.csv".

How I can use a counter so that next time when the job will started again
the output file would look like this "List01.csv".

And next time again "List02.csv" etc.

I tried to use tJavaFlex and the following code

int count= Integer.valueOf(counter);
context.Counter=count+1;
System.out.println(count);

but it's not working.

Or can I set a global variable which will increase?

Thanks for some ideas/hints.

Re: delimiter output file with increasing file name

Anonymous — Thu, 20 Jun 2019 15:54:29 GMT

Hi @pawe84

Whenever you start the job the global variables/context set earlier will be flushed.

Hence the only way is to store the job count in a file, and in the pre job read the file/Table, add one more to the job count and store to context/Global Variable and use the same context/global variable to name the file.

After that again update the file/table with new value of the counter.

Thanks and Regards,

Subhadip

Re: delimiter output file with increasing file name

pawe84 — Tue, 02 Jul 2019 09:21:39 GMT

Now I tried to use tFileInputDelimited where I read a file with my star value.

Then the next component is tJavaRow with the following listing

context.count=input_row.seq;

context.count++;

But how I can get context.count now in order to save it tFileOutputDelimited? Which componend I have to use?

If I use tJavaRow again: output_row.seq=context.count it doesn't work I'll get an error.

Re: delimiter output file with increasing file name

pawe84 — Tue, 02 Jul 2019 09:41:46 GMT

Okay, now I got it.

I used tFixedFlowInput.

Is there a way tom reset the counter if a new day/ date is occuring?

E.g: Today the job was started five times --> stored value=5

When the job will be started tomorrow that there's a reset --> stored value=0 ?

Re: delimiter output file with increasing file name

Anonymous — Tue, 02 Jul 2019 19:22:27 GMT

Hi,

The best way is to store the date and running value to a database table (if not available, then use a control csv file).

It should have two columns i.e., the date and current file sequence number. So at the beginning of your job, you will have to compare whether the date in the table(or file) is same as current date. If the condition is wrong, you can start the job with initial value i.e. 0.

If the condition for date is correct, you can assign the current value+1 to a context variable and use this value while creating the file name.

Once the process is successfully completed, you need to insert both current date and current context value back to DB (or file) so that you can repeat the same process next time.

Warm Regards,
Nikhil Thampi

Please appreciate our Talend community members by giving Kudos for sharing their time for your query. If your query is answered, please mark the topic as resolved 🙂