Unlock a world of possibilities! Login now and discover the exclusive benefits awaiting you.
Hi, thanks for the response. that would be great. i will be waiting for your response. i am trying my part by coding.,but i want to make use of talend. so i will really appreciate if you can share it once you are done. Regards.
SCD in Hadoop can be implemented using Hive ELT components. My implementation has three parts:
-Change Data Capture (Using tELTHiveMap, tELTHiveInput(s) and tELTHiveOut). The SCD table and staging table that contains today's records need to be left joined on the keys and if record exists compare the columns and write the appropriate code (Insert, Update/Insert). I wrote the results to a Delta Table.
- Now Apply the Inserts (New rows).
- Apply the type II updates (or SCD). Insert Overwrite the rows that needs to be obsoleted. Add the new version needs to be updated.
-Change Data Capture (Using tELTHiveMap, tELTHiveInput(s) and tELTHiveOut). The SCD table and staging table that contains today's records need to be left joined on the keys and if record exists compare the columns and write the appropriate code (Insert, Update/Insert). I wrote the results to a Delta Table.
- Now Apply the Inserts (New rows).
- Apply the type II updates (or SCD). Insert Overwrite the rows that needs to be obsoleted. Add the new version needs to be updated.
Hi
sdhurjati,
Thanks for your response. i am trying to download the code .,but it is saying page not found. can you please re-upload the zip file again. i would be grateful to you. i am trying the process you explained now. i will wait for the zip upload. thanks a lot.
Thanks for your response. i am trying to download the code .,but it is saying page not found. can you please re-upload the zip file again. i would be grateful to you. i am trying the process you explained now. i will wait for the zip upload. thanks a lot.