Unlock a world of possibilities! Login now and discover the exclusive benefits awaiting you.
We have a requirement in our Big Data project where we need to validate around 1 Million records
We have a source SQL to retrieve data from teradata table and a target SQL to retrieve data from Hive table and then do a row/row comparison but the issue was - the Job aborts when the number of records pulled from Teradata exceeds 200K.
Is there any way to pull and validate around 1M records ?
Please refer to the attached screenshot for the error details.
Hello,
Would you mind posting your job design screenshots on forum which will be helpful for us to address your issue?
Best regards
Sabrina