topic Re: logic for date on batch processing in Talend Studio

logic for date on batch processing

mahesh_Dev — Sat, 16 Nov 2024 07:27:28 GMT

hi all,

i need logic to develop from source data that job should read starting date and till ending date and this should be processed

example

we are running batch job on 12:00 till next date

15/10/2018 12:00 till 16/10/2018 12:00

i need the logic how can we achieve

thanks

Re: logic for date on batch processing

Anonymous — Tue, 16 Oct 2018 12:28:59 GMT

Hi,

My understanding is that you are trying to run a delta data extraction process. If you are using a Database, the advise will be to store the last run time in the DB and pick the last run time from DB as the first first step. You can store this data into a context variable and continue the processing. In this way, you will not miss any data if you are manually running beyond the normal schedule time.

Now, if you want to capture the 12 am time of current day, you can create a context variable and populate the current day to it (using Talend or java date functions).

If the answer has helped you, could you please mark the topic as resolved? Kudos are also welcome 🙂

Warm Regards,

Nikhil Thampi

Re: logic for date on batch processing

mahesh_Dev — Wed, 17 Oct 2018 05:40:37 GMT

nikhil

could you help me how to do this

thanks

Re: logic for date on batch processing

Anonymous — Wed, 17 Oct 2018 05:50:59 GMT

Hi Mahesh,

I would say, try to create a sample job based on the above details and if you are stuck, please post the job screen shot and other details in the forum. In this way, you will learn the finer aspects of Talend tool yourself 🙂

Also talk to your architects and designers about how they want to fetch the delta data. Give all the options discussed above with them so that they can take an informed decision on job creation.

Warm Regards,

Nikhil Thampi