i am new to talend. While understanding Data Insertion in Talend , come across a doubt.
While inserting Data by using toracleoutput or any other bulk components, due to any reason if Connection failure is happen how the data will be committed and how system knows to process remaining rows in the next re-run.
"Commit every": define number of rows to be completed before committing batches of rows together into database. "Batch Size": Specify the number of records to be processed in each batch.
If connection failed/job failed, then on re-running the job, it will re-insert the committed records to oracle table. If need to prevent this, need to put tOraclerollback in after the subjob runs.
It is your job as a data integration specialist to build jobs that will support that. The tools in Talend are there, but there is no magic button to do it for you. As an example, you could keep a log table to log success/failure, number of rows, watermarks (high/low) of your source data, etc. When your job starts you look at this table to get history information, then write to this table to acknowledge a start. Then when your job finishes successfully or with a failure, you store information in that table to help you with your next run.
"Commit every": define number of rows to be completed before committing batches of rows together into database. "Batch Size": Specify the number of records to be processed in each batch.
If connection failed/job failed, then on re-running the job, it will re-insert the committed records to oracle table. If need to prevent this, need to put tOraclerollback in after the subjob runs.
