Partial data Load from one schema to other schema through Talend Job
Currently we have created a merge statement script in snowflake and calling it in Talend job to load data from TRNS schema to RAW schema table.
Its good at performance standpoint, but now we need to achieve Partial data load in this case if at all in errors comes during merging the data and as well as need to get rejected rows too in Talend job.
We have table A created in DB schema X. need to load table B in DB schema Y. A and B having same number of columns and data type. currently datatype is matching in both the schemas for table A and B for column to column. but it may change get in future.
Created a Talend job to load data in Table B from table A by using merge statement. It is Update Insert operation while loading data in table B on the basis of primary key.
A procedure has been developed in snowflake database which includes Merge statement of tables. This procedure is being called in tDBRow_1 component in Talend Job.
