I have a QVD with data from Jan 04th 2016 until last Sunday. Since it does not make sense, to read the entire values from Jan 04th 2016every week simply the last week is added to the existing QVD (with Concatenate command). So far so good.
Now I would like to read not only the last week, but Sunday again, because for this there are mostly still missing data, which are only gradually recorded during the week. That's why I have a lot of duplicate records that I didn't need in the QVD because this indeed falsify values.
Is it possible to detect and omit these duplicates when reading the data by the script or afterwards? And if so, how?