I have concerns about the required logic here. You have, in your pseudo code, a line that says....
check every 15min continues(infinity loop)
Talend jobs are batch. They should start, process and end. It is not a great idea to leave them essentially polling every 15 mins. To do this you may want to use Talend ESB.
The other concern with this is the expectation that your dates will match exactly. Are they dates or Strings representing dates? If it is the latter, then this will work (although don't compare them with "="). Otherwise you may want to truncate your dates to just days, since the whole date will go down to milliseconds.
However, before you get to the actual comparisons, you do need to think about whether you have to have a job that *could* run forever. What is the business the requirement you are trying to solve?
Yes, it is business requirement. Whole job i cant share if want share any gmail ID i can.
Coming to point 1.
I have concerns about the required logic here. You have, in your pseudo code, a line that says.... check every 15min continues(infinity loop)
Talend jobs are batch. They should start, process and end. It is not a great idea to leave them essentially polling every 15 mins. To do this you may want to use Talend ESB.
Ans: Let me check with Talend ESB. Any specific component in ESB i need to work around.
The other concern with this is the expectation that your dates will match exactly. Are they dates or Strings representing dates? If it is the latter, then this will work (although don't compare them with "="). Otherwise you may want to truncate your dates to just days, since the whole date will go down to milliseconds.
Data type is Date and Oracle DB.
Please let me know any further input or if you have any question?
With regard to the ESB you will want to look into Routes. Do a bit of reading on Apache Camel routes. It isn't as straight forward as the ETL tool though. You will be wanting to look at this http://camel.apache.org/sql-component.html
Another way you could approach this (which would be safer if using an ETL job) is to schedule the whole job to run every 5 or so minutes. This would remove the need for using the ESB, but would make the job a lot safer. Something to consider. This really does depend on your business requirement I guess.
If you are using Oracle, you can output the Date data truncated to yyyyMMdd which would solve potential matching issues.
An easier way (but maybe a little bit more restrictive) would be to create the job and not worry about the 15 minute looping logic inside the job. Handle the looping using a scheduling tool. Just call the job every 15 mins.
How to achieve looping here. any specific components?
Can i use tPostjob and tPrejob to achieve this logic?
if both table date matches then create temp file and using twaitforfile components wait for that file until once is received run the 2nd subjob and in tPostjob delete the temp file which created after match condition.
It is not a good idea to have a Talend job running forever. They are not meant for that. So what I am suggesting is that you create a job to test your logic BUT don't include the waiting functionality. So the flow would be like below....
Scenario:
1) In between tOralceOutput1 and tOralceOutput2, Need to check below logic.
if table_1.date_col <> table_2.date_col
wait for process to call ( tOracleInput ------>tOralceOutput2(table_2)
check every 15min continues(infinity loop)
else
if table_1.date_col == table_2.date_col
continues to next step ( tOracleInput ------>tOralceOutput2(table_2)
end if
Instead of checking every 15 mins within the job, schedule the job to be run every 15 mins.
