Hello,
I need to compare tables in Impala vs table in SQL server. I am currently using tmap component to compare the tables.As we have 200million records to compare it is taking around 2 days to complete.
My flow is as below :-
tsqlinput and tjdbcinput connected to tmap and results in excel.
Is there any other approach in talend to compare impala tables vs sql server to complete the comparison faster.
Regards,
Raakesh R
i would suggest sqoop to move the sql data to the cluster and then doing the comparison on the cluster. that should allow you to scale the job and radically reduce your process time.
