4 Replies Latest reply: Sep 24, 2012 11:20 AM by Srinivas Aremanda RSS

    Missing data based on load order

      Depending on the load order of various databases in QlikView, we seem to be missing portions of our data out of some of them.

      Example:

      Run QVW files in the following order:

      1. QVD output to file:
        1. General Ledger
        2. Enterprise system
        3. Government certification requirements
      2. "Database builder" - takes all QVD's and reads them in, along with a few mapping connections and account security settings
      3. Dashboard - loads DB builder from a binary and contains charts, graphs, and KPI's

      and all of the data is there, but run the files in the following order instead:

      1. QVD output to file:
        1. General Ledger
        2. Government certification requirements
        3. Enterprise system
      2. Database builder
      3. Dashboard

      and somehow some of the departments are missing from the certification requirements data.

      We've also seen (related?) that sometimes if we add a new chart to our dashboard that counts/calculations will be wrong on other charts in the dashboard and the dashboard file size will be smaller than it had previously been (before the new chart) until we reload the database builder and the dashboard again.

      Has anyone else seen this? Have an explanation?

        • Missing data based on load order
          Rob Wunderlich

          So the only difference I see is the order of QVD builds in step #1, correct? Can you post the script for Step #1?

          If you can't post the script, examine the use of exists(), join, drop -- these are all items that could be affected by the order of things.

          -Rob

            • Missing data based on load order

               


              Rob Wunderlich wrote:
              So the only difference I see is the order of QVD builds in step #1, correct? Can you post the script for Step #1?
              If you can't post the script, examine the use of exists(), join, drop -- these are all items that could be affected by the order of things.
              -Rob<div></div>


              Thank you so much for your help. Each portion of #1 is actually a separate QVW file being loaded with many separate scripts creating many separate QVD's as its output. None of the QVD creators in step #1 have any dependencies on the others, to my knowledge. Table viewer in our dashboard looks a little like a plate of spaghetti with 27 tables in it and there are probably about 60-ish pages of load script altogether in the QVD creators.

              As far as I am aware, nothing that's loaded from QVD into the database builder is dropped as a full table (just individual fields) and there are no joins or exists()'s in there, but I'll take a look and post back if I'm wrong.

            • Missing data based on load order

              Hi Holly M.

              I know this is an old thread but it sounds very similar to intermittent Data Loss we are experiencing.

               

              I've just replied to someone elses recent post here http://community.qlik.com/message/197579 which explains what we are seeing.

               

              In short I have created a document that runs on the QV server, loads data from various tables in a SQL database, writes each result set to a QVD file, then checks the count of records that were loaded into the QVD's.  Our source database isn't changing so we expect the record counts to be constant.  Unfortunately we are seeing random data loss - sometimes of only a few records sometimes over a thousand.

              We have tried a number of things but nothing has resolved it.

              Did you manage to resolve your problem?

              Andrew