3 Replies Latest reply: Aug 11, 2015 12:04 PM by Richard Garforth RSS

    Disk Striping in QlikView

    Shane Spencer

      We have particularly large documents. Some are 5GB on disk, 25-30GB in Memory and use over 40GB of Memory for the reload. We have over 2TB of SAN disk attached for QVD files and QVW's.

       

      I was wondering if the SAN attached disk would benefit from being striped? Like we would do on a SQL / Database server.

      Has anyone got any experience of this?

        • Re: Disk Striping in QlikView
          Bill Britt

          Hi Shane,

           

          I have heard of this for backing up the data, but nothing else.

           

          Bill

            • Re: Disk Striping in QlikView
              Shane Spencer

              Thanks Bill. I couldn't find anything on google but we have some pretty large / long-running reloads that read/write to disk simultaneously. I've been looking at the Distributions and some are taking 20-30 mins for the Distribution alone. I was wondering if slow write speeds were a contributing factor. Disks striping allows more concurrent reads/writes to Disk, that why it's recommended for databases.

                • Re: Disk Striping in QlikView
                  Richard Garforth

                  Shane,

                   

                  The modern Storage devices will already be striped under the covers, what you need to concern yourself with is the concurrent usage of the Storage with other users of the Storage array, or common components of the SAN. 

                   

                  Take a look at the disk trhoughput metrics on the server, are they flat lining at any point, indicating contention?  Are they faster one day to the next, indicating other load on the common components?  Are you connecting to your device via Fibre channel connectors, or are you actually connecting over the network interface?

                   

                  Striping on Databases works because the database software effectively splits the reads processes up parallely (sp?) and each parallel process requests a unique block from the database.  if the Database block size and the Stripe size of the database are in sync, then a 4 way striped volume with a paralelism setting of 4 will send 4 concurrent requests to the storage, and this minimises delay.