4 Replies Latest reply: Mar 11, 2015 1:48 AM by Nirmal Azhagarselvam RSS

    Rank() in Hadoop


      This might not be a right question to ask here but Hadoop forum did not help me either.

      We need to filter records from Hadoop (source) table before loading it to a QVD since volume is too big to load as it is from source.

      In the existing script they have used Rank() and it was working because it was fetching data from oracle. But when I tried to replicate Rank() in Hadoop it keeps throwing generic error. I do not have any experience in Hadoop and do not have client to execute SQL and check. If anybody tried hadoop analytic function before please help me out. It would be great if you share any workaround.


      Sample data:

      ServiceID      Date          Time

      A                12/14/2014   2:13 PM

      A                12/14/2014   3:15 PM

      B                12/15/2014   1:15 PM

      B                12/15/2014    2:20 PM  


      I need to bring only 2 records which are highlighted above (recent records).