Hi Team, I want to know what are the options and components available in Talend Open Studio for Data Integration that can be used in testing of data migration. Regards, Asanka.
Hi,
Do you want to monitor row In/Out during your Data Migration Job executing? Would you mind giving us more description about your requirment?
Best regards
Sabrina
Hi Sabrina, In brief I need to verify source data has been successfully migrated to the destination. This verification needs to be carried out after the migration of particular subset of data in other words after completion of migration job. This needs to be done as a part of migration test in order to verify the data migration. This migration testing should ensure that transfer of data from sources db to the target db occurs without data loss, with strict adherence to transformation rules, and in compliance with all validity checks. Thanks and Regards, Asanka.
Hi Sabrina,
Yes. That is the same topic what I expected. In it says that 'Talend Open Studio for Data Integration makes ETL testing easier and more effective'. Is there any standard approach or procedure to follow in ETL testing using Talend. Please share any resources if you aware of this.
Thanks and Regards,
Asanka.
Hi,
With Talend Enterprise Subscription Version, there is CDC feature(change Data Capture) which capture only the changed source data and to move it from a source to a target system(s) in real time.
TalendHelpCenter:CDC architectural overview With Talend Open Studio, you have to compare Source DB and target DB.
Best regards
Sabrina
Hi Sabrina, Thank you very much for your quick response. In my case it doesn't need to capture change data in real time. Data migration from source db to target db will be a one time process. What I need is to ensure source data has been successfully migrated to the target db according to the predefined mapping rules without any data losses. Is it possible use talend open studio for this purpose. Thanks and Regards, Asanka.
Hi, You can compare target DB and source DB to check if there is any missing data. For example: TargetDBTable--main-->tmap-->--->tlogrow SourceDBTable--lookup In tmap, please choose inner join for your input data and set "Catch lookup inner join reject" as true. If there is no output data when executing job, no missing data during Data Migration processing. Best regards Sabrina
Hi Sabrina, Thank you very much for the suggestion. I'll try it and post here the result sooner. I highly appreciate your quick responses and feedback. Thanks, Asanka.
