I have a 3 layers of loading data.
Layer 1- I extract data from the SAP and store it in a QVD.
Layer 2 - Iam transforming all the QVDS from Layer 1 and storing back again in another QVD.
Layer 3 - Data model and Dashboard.
Now here in Layer 1 - iam having 2 ways to load ..insert button which load the whole data and update button which loads last month data..and iam flagging it as insert or update based on the button
Now in layer 2 i have again insert and update button. i.e do the xxx transformation if it is insert ,else if it is update do the yyy transformation.
now my question should I make all the data to insert on the raw data after the layer 2 has run?
