I am comparing(Data validation row by row comparison) two tables in hadoop database and SQL servers using tmap component.
The HDFS file size - 30.3 GB and
Record Count is - 328268683. Error is as below :- There is not enough space in hard disk.
Configuration - 64 bit ,8 GB RAM,64GB hard disk.
Please let me know incase of any other alternatives other than tmap.
Erro log
Starting job ImpalaDemoFA_SS_BarcodeTransaction_QA at 11:30 18/06/2015.
connecting to socket on port 3623
connected
java.lang.reflect.InvocationTargetException
at sun.reflect.GeneratedMethodAccessor1.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
at java.lang.reflect.Method.invoke(Unknown Source) at org.jboss.serial.persister.RegularObjectPersister.writeSlotWit
Hi,
Have you already stored temporary data on disk?
What's kind of comparison? Could you please give us more description about your job requirement so that we can see if there is
any other alternatives.