5 Replies Latest reply: Jul 4, 2012 9:17 AM by Abhijit Bansode RSS

    loading multiple QVD files to the same table

    mbagge

      So I am here with some usage data fetched from database and stroed in QVD files (one per month) and want to read the data back to a QVW.

      I tried to read them in a schedule fashion where I only want the three latest months data:

      UserUsage:
      LOAD * FROM statData\$(firstyear)-$(firstmonth)-UserUsage.qvd (qvd);
      LOAD * FROM statData\$(secondyear)-$(secondmonth)-UserUsage.qvd (qvd);
      LOAD * FROM statData\$(thirdyear)-*-UserUsage.qvd (qvd);

      (firstyear, firstmonth and so on are populated with the actual years and months according to a calculation earlier in the script)

      When executed I get this in the log:
      UserUsage << 2009-05-UserUsage 152242 lines fetched
      UserUsage << 2009-06-UserUsage 181930 lines fetched
      UserUsage << 2009-07-UserUsage 211618 lines fetched

      It reads the files alright but only 211618 lines are noted in document proporties, more in the line of 545000 was my wish.

      So I thought I change the code a bit.

      UserUsage:
      LOAD * FROM statData\$(thirdyear)-*-UserUsage.qvd (qvd) WHERE usageType = 1 OR usageType = 3 OR usageType = 15;

      here I load the last years Usage data, will not be too useful the first month in a year but that's another problem. The log looks more or less as the one for three months moving but starting with 01, 02 and so on. The result is however the same.

       

      How do I append the data from multiple QVD files to the same table? I can't do that? Then I have to reread the complete database every day and that doesn't seem like a good idea at all.