I had designed a talend job (in screenshot "Parent_job"), which has a sub job called DT_P_Oracle (in screenshot "Sub_job"). In the sub job I had used 4 tOracleInput where I had divided the month in where clause. Some records randomly missing in table (tOracleInput) read itself (once in a week) which is random in any one of the four sub jobs in screenshot "Sub_job". Please provide solution for this
We do not have enough information to help you. Can you give us the SQL queries with where clause. tOracleInput read itself ?what do you mean? What is your commit method? I did not see any parallel configuration into your job?
I had attached tOracleInput query in "select Query.txt" file, where I had changed "dt.posted_date" values in tOracleInput_1, tOracleInput_2, tOracleInput_3, tOracleInput_4.
"tOracleInput read itself ? what do you mean?" -> I am trying to say that the records (nearly 80 Lakhs records of the total - 5 crores) were missed during the reading process in Oracle DB (tOrackeInput) sometimes. But most of the time, it works well.
"What is your commit method?" -> I had attached the screenshot "commit method".
"I did not see any parallel configuration into your job?" -> I had enabled multi-thread execution in the main job, and four sessions were opened at back-end at a time for reading data.
when SBITABLETRUNK had run DT_P_Oracle is runing I think SBITABLETRUNK trunk table. DT_P_Oracle is configured to run 4 threads. Is there a configuration to split the job? or are they doing the same job? I think your problem is here!
There's no configuration to split the job except multi-thread execution, All the four threads were doing the same job with a difference in where clause's posted_date range (refer "select Query.txt" in previous attachment).
will there be any problem in multi-thread execution?
I had split the four sub-jobs and used tParallelize with four tRunJob. still, 47 Lakhs records missing in the 2nd job (this time), do you have any other suggestions
