8 Replies Latest reply: Jun 7, 2010 10:08 AM by jgerritsen RSS

Best practive for version control/collaborative development with QlikView

Hi to all,

for some days I am now working on a best-practice-example for using version-control in QlikView.

Assume that one is using subversion or a similar versioning-software (CVS-system) and would like to cover the following issues in a larger development team:

  • Versioning and collaboration of Load-Scripts
  • Versioning and collaboration of user-interface-development within the same QlikView-application (includes macros)
  • Test and production-environment
  • (in an ideal world): automated testing

What are your experiences, best practices, suggestions?

Looking forward for the discussion in this thread!

Best regards

Stefan

  • Best practive for version control/collaborative development with QlikView
    John Witherspoon

    We don't have enough developers to have serious issues, so I won't be much help, but I'll share as much as I can because I'd like to see what we can all come up with and perhaps adopt some best practices.

    We have three environments - development, test and production. In practice, we do all of the development AND testing in the development environment. We only move to test as part of the install procedure, so it would be better called "staging" at the moment. But the idea of the test environment would be that all of the QVDs mirror production (they don't in our system), you point anything else at your production data, and you would retest everything one last time there before finishing the install.

    We do have a code versioning system, but are only using it as part of our install process, not for development. For our more critical applications, when I install them, I also put a manually versioned copy (just a number on the end) in an "archive" folder on the test server. I don't believe this is a policy; it is just something that I do because I tend to be cautious. Similarly, I also keep manually versioned copies of the code while I'm developing. This has saved me at least once in the past, I believe twice, when the file I was working on became corrupted, and I'd already saved more than the automated number of versions.

    I've written automated testers, but not for QlikView. It hadn't really even crossed my mind to do it for QlikView, and yet it would be awfully nice to have for some of the more critical applications. Our regression testing procedure is currently loose at best. Pretty much if your new feature works, and nothing jumps out at you as broken (no missing charts), you install it. If you find you've broken something (so far thankfully rare), you fix it and install it.

    Most of our applications load entirely from QVDs. We could store off a version of the QVDs to run the automated testing on. Now, how to test? Well, you want to test that the script still loads up your data model properly. Seems like you'd want to load up a lot of meta data - what are the fields, how many distinct values for each, what is the average value for numeric fields, that sort of thing. You'd need to store the expected values somewhere. It would be nice to just export this from a known working version of the application, so maybe a table with all this data that you can export to Excel? Then you want to make sure the charts are working. How much can you reference through the API? Can you loop through the dimension values and check the results somehow? This is all a bit beyond me at the moment. Again, you definitely want to be able to export the desired results somehow rather than having to manually enter thousands of different numbers, and then if you change some reports intentionally, fix thousands of numbers.

  • Best practive for version control/collaborative development with QlikView
    mbagge

    FYI, I use Mercurial for VC.

     


    Stefan WALTHER wrote:<ul><li>Versioning and collaboration of Load-Scripts</li><li>Versioning and collaboration of user-interface-development within the same QlikView-application (includes macros)</li></ul>

    I am the sole developer right now so the collaboration aspect is not covered but I try to end my development days with committing the state of the scripts as separate text files just to have the history at hand.
    I have the QVW files in my repository too just for the sake of it.
    I really want good advises for how to deal with the VC part of the development process because it feels like my spine is trying to leave my body every time I have to move code from QV to Emacs just to make the VC some part of the project. And doing "some small changes" without being able to do a clean start is also annoying. For normal development I tend to commit after more or less all changes just to ?eep track of what I am doing, this is non existant in QV and doing reverts is really really painful.

    Stefan WALTHER wrote:
    Test and production-environment

    Production is an isolated part, all other things are done at my workstation running Windows XP.

    Stefan WALTHER wrote:
    (in an ideal world): automated testing

    No, nothing at all for QV.

      • Best practive for version control/collaborative development with QlikView
        Steve Rimar

        We have QlikView deployed to several development teams and use different solutions based on each team's requirements.

        The most widely used solution is to use Subversion for version control along with Tortoise SVN which is an integrated plug-in for Subversion through the Windows explorer. The only issue we have with Subversion is that it's meant for doing commits and merges. Unfortunately you can't merge two QlikView applications. As a result, what we end up doing is forcing the Subversion repository to ensure a lock is placed on a file before a change is made to it.

        So, our Subversion workflow looks like this:

         

        • SVN Update - basically Subversions way of saying get the latest version
        • SVN Get Lock - lock the QVW file so that others can't make changes. Think of this as a check out.
        • Publish - this is a step that is optional and not part of Subversion. By publish we mean publishing the QlikView application to our server. You wouldn't always do this, only when you need to promote your changes to the server.
        • SVN Releaes Lock - this releases the lock so that you can do the final step...
        • SVN Commit - this commits the QVW to the SVN repository so that when the next person does an SVN Update they get the latest version.


        You could improve this process by adding custom scripts to your Subversion repository when a command is issued. For example you could merge the Release Lock and Commit into one. You could also add a command to publish to the QlikView server, but this would be an advanced feature.

        Additionally, we're also looking into utilizing SharePoint. SharePoint would provide some nice approval workflows and much more intuitive versioning capabilities than Subversion for our Professional users.

        • Best practive for version control/collaborative development with QlikView

           


          steverimar wrote:
          We have QlikView deployed to several development teams and use different solutions based on each team's requirements.
          The most widely used solution is to use Subversion for version control along with Tortoise SVN which is an integrated plug-in for Subversion through the Windows explorer. The only issue we have with Subversion is that it's meant for doing commits and merges. Unfortunately you can't merge two QlikView applications. As a result, what we end up doing is forcing the Subversion repository to ensure a lock is placed on a file before a change is made to it.
          So, our Subversion workflow looks like this:

          <ul><li>SVN Update - basically Subversions way of saying get the latest version </li><li>SVN Get Lock - lock the QVW file so that others can't make changes. Think of this as a check out. </li><li>Publish - this is a step that is optional and not part of Subversion. By publish we mean publishing the QlikView application to our server. You wouldn't always do this, only when you need to promote your changes to the server. </li><li>SVN Releaes Lock - this releases the lock so that you can do the final step... </li><li>SVN Commit - this commits the QVW to the SVN repository so that when the next person does an SVN Update they get the latest version. </li></ul>

          You could improve this process by adding custom scripts to your Subversion repository when a command is issued. For example you could merge the Release Lock and Commit into one. You could also add a command to publish to the QlikView server, but this would be an advanced feature.
          Additionally, we're also looking into utilizing SharePoint. SharePoint would provide some nice approval workflows and much more intuitive versioning capabilities than Subversion for our Professional users. <div></div>


          Hello Steve,

          thank you for your detailed answer ... very interesting ... I am on a very similar way during research/development/customization for two of my clients ...

          I'll come back here later on with my findings in detail :)

          Best regards
          Stefan

          • Best practive for version control/collaborative development with QlikView
            jgerritsen

            Last Monday QlikView announced a new partnership with NOAD, a company that sells a Version Control package for SAP/BusinessObjects. They have now ported their solution to QlikView and I'm planning on starting an evaluation to see what specific QlikView functionality they have developed.

            Best regards,

            Jeroen

            • SV:Re: Best practive for version control/collaborative development with QlikView
              GandalfGray

               


              Jeroen wrote:
              Last Monday QlikView announced a new partnership with NOAD, a company that sells a Version Control package for SAP/BusinessObjects. They have now ported their solution to QlikView and I'm planning on starting an evaluation to see what specific QlikView functionality they have developed.
              Best regards,
              Jeroen<div></div>


              Hi Jeroen

              How is your evaluation going?

              It would be interesting to here if you found it useful! Smile

               

              Apart from the basic tracking of changes, what we need at our site is good merging capabilities of the version control tool.

              Best Regards

              /Göran

               

              • SV:Re: Best practive for version control/collaborative development with QlikView
                juergm

                Hi

                We can probably have the load scripts separate from the GUI functions but chances that the app will break are rather high. I would therefore not try to collaborate on a single app.

                For testing I haven't seen much in this thread so far. My approach would be:

                a) create some reference data in your database or files. Make sure you can recreate it

                b) define a number of scenarios and define the expected result

                c) create bookmarks for each scenario

                d) Not sure about this step but it might be possible to create a macro stepping through the bookmarks and create Excel/PDF output of the sheets. Another option might be to use AutoIt or antoher automation tool to control the QlikView app. This tends however to be rather difficult to achieve and maintain.

                e) make sure your boookmarks and expected results match and create a reference for future regression tests.

                e) The professional version of Acrobat has the option to compare PDF's and will point out differencies in a very nice manner. Unfortunately this is a rather expensive solution. Also Excel-Comparison is not as easy. Solution might be to create CSV output which can be easily compared even over folder trees.

                Juerg

                 

              • SV:Re: Best practive for version control/collaborative development with QlikView
                jgerritsen

                Hello Goran,

                The evaluation went quitte well. NOAD EQM add's Version Control capabilities to our QV Development process making it easier to work with multiple developers. Rolling our QV documents to our QA and Prod environments is also controlled and audited. Although we have some Enhancements for new releases it's already a usefull product for our organization. This version does not provide merge capabilities as far as I know.

                If you want to see it for yourself you can join one of the webinars on www.noadbi.com. I would be interested in your opinion as well!

                Best regards,

                Jeroen