Hello. I have a question regarding Talend's SCD component behavior in case of multithreaded execution.
Imagine the following situation: there is an input table with a business ID and some other columns. To simplify things, let's imagine that the input table has three columns: "business ID", "Name" and "Date".
I'm going to use the SCD component to create a surrogate key (to be independent of the format of "business ID" field) and to track the history of changes of the "Name" field. I'm also going to process the input table in parallel using the tFlowToIterate component. And finally, I'm going to use the "Date" field as the "SCD start" parameter of the SCD component.
Let's imagine that the input table contains two records:
business ID: 123, Name: John, Date: 1 January 2019
business ID: 123, Name: Chris, Date: 2 January 2019
I'm worried that in the case of multithreaded execution things can go wrong. Imagine the following process:
1) For the first record in the input table, the SCD component asks the question "are there any records in the DB which correspond to business ID 123?" The DB responds "no, there are none of these records". Talend then generates a surrogate key "id = 1" corresponding to business ID 123. And then the OS decides to switch thread
2) For the second record in the input table, the SCD component again asks the question "are there any records in the DB which correspond to business ID 123?". And again, the DB responses "no, there are none of these records", since the first thread had no chance to write anything to the DB. Talend then generates a surrogate key "id = 2" (or any different from "id = 1") corresponding to business ID 123. And then the OS decides to switch thread again
3) We end up with two surrogate keys for the same business ID in the SCD table.
The question: is the described situation possible in Talend?
