2 Replies Latest reply: May 25, 2012 4:47 AM by Magnus Åvitsland RSS

    Issue of Incremental Load (Where not exist)



      My source data from one particular view gives exactly identical rows for key column (Say order id)


      When I read it into QV, though it is not displaying multiple rows, I do see correct value if I apply aggregate function on numeric column (eg. Qty)

      Also it loads all rows into QVD.


      My problem occurs while doing incremental load. Due to data nature, I can’t use date filter and load only new set to QVD. I must use Where not exist (order id) to handle incremental load (Combine current data + (QVD data – Current data) and load back into QVD).


      When I do that it reads only one row from QVD and eventually I lose those ‘duplicate’ rows from qvd.


      One solution could be applying Aggr while reading data from source. But seems user wants to see ‘n’ number of rows as in source.

      Do we have any work around?