My understanding is that you are trying to run a delta data extraction process. If you are using a Database, the advise will be to store the last run time in the DB and pick the last run time from DB as the first first step. You can store this data into a context variable and continue the processing. In this way, you will not miss any data if you are manually running beyond the normal schedule time.
Now, if you want to capture the 12 am time of current day, you can create a context variable and populate the current day to it (using Talend or java date functions).
If the answer has helped you, could you please mark the topic as resolved? Kudos are also welcome 🙂
My understanding is that you are trying to run a delta data extraction process. If you are using a Database, the advise will be to store the last run time in the DB and pick the last run time from DB as the first first step. You can store this data into a context variable and continue the processing. In this way, you will not miss any data if you are manually running beyond the normal schedule time.
Now, if you want to capture the 12 am time of current day, you can create a context variable and populate the current day to it (using Talend or java date functions).
If the answer has helped you, could you please mark the topic as resolved? Kudos are also welcome 🙂
I would say, try to create a sample job based on the above details and if you are stuck, please post the job screen shot and other details in the forum. In this way, you will learn the finer aspects of Talend tool yourself 🙂
Also talk to your architects and designers about how they want to fetch the delta data. Give all the options discussed above with them so that they can take an informed decision on job creation.
