Source Control and multiple developers question

[stevelord]

Hi, my own situation is pretty simple and we don't have many people working on the same file at the same time, but a colleague is trying to work stuff out for their team and I thought I'd bounce the problem off the wall here.  We have TortoiseSVN/CollabNet something and stuff.

Scenario: one file, two developers.  Each developer gets latest version of the file.  Developer A makes changes to page 1 and checks those back in while Developer B is making changes to page 2.

Question 1: How does Developer B check in their changes to page 2 without overwriting Developer A's changes to page 1?  Or how can Developer B get Developer A's changes onto their file before checking in their change file, without overwriting their own changes in the process?

Question 2: If the production Admin person is a different being from the two above, and they are checking in changes, is there a way for them to confirm the changes they are checking in mesh with what the developers have noted in their log, or are they trusting the changes the developer noted are the only things going in as intended?

Does source control recognize that Developer B only edited page 2 and only try to check that in without overwriting what Developer A did for page 1? Or is the file updated by Developer A simply being replaced by the copy of the file updated by Developer B?  I haven't had a chance to test, but it seems to me source control ought to be able to check in only what a developer edited and not mind if the rest of the file happens to be different since those differences are likely changes another Developer had made.  (And maybe the resolve conflicts step is needed if the one developer turned a chart green and another turned the same chart blue.)

Any insights are welcome.  Knowledge that benefits the colleague may benefit me down the line.   (As it is, it's just a clever way for me to get notes next to changes I check in, and maybe recover or revert files I screw up.)

[stevelord]

question 2 admin 'checking in changes' relates to admin pulling latest copy of the file down to the production/server folder.  Developers checking in their changes to the repository from their workspaces, and the admin pulling the changes from the repository down to update that file on the server.

[Miguel_Angel_Baeyens]

Hi Steve,

Source Control does not use QVW files, that are binary and unlikely to handle for this purpose, but XML files that are created in a -prj folder right next to the QVW file they belong. This XML files are the ones integrated in the source control and therefore you do can even change a chart in Sheet1 and another colleague modifying another chart in Sheet1 as well.

To see how it behaves, let's pretend you have a file Dashboard.qvw and you create a new empty folder named Dashboard-prj. Then open the file, save it and see the contents of the -prj folder.

I'd strongly recommend to take a look at this document on the subject: Updated Guide to QlikView and Source Control (TFS & Subversion)

Miguel

[rwunderlich · Accepted solution]

As Miguel has noted, you don't manage qvws with Source Control, you use the files generated in the -prj directory. The idea is the -prj files represent granular app elements, and the text nature of the prj xml files will facilitate diffs and merges.

The paper that Miguel references is essential reading.

I have found the prj facility challenging for use a multi developer solution. The logical file relationships and the quantity of xml is more complex than a typical source project like java or a web site. Let me give you some examples that will address some of your questions.

1. Each sheet object creates a separate objectid.xml file. This is great, as your chart and my new chart are in separate files. However, the entire app shares a single QlikviewProject.xml file. This file list all sheets and objects and their positions. So if both of us add independent objects (files), we will have a guaranteed conflict on the QlikviewProject file. The file is xml -- lot's of it -- so unless your tools are xml aware the merge can be a real pain. If you screw up the merge, you won't be able to open the qvw.

2. The prj process is too "chatty" in my opinion. A save will update comments in files like "last updated on" which looks like an update and if your tools are not properly configured to ignore comments, it forces an update and possibly a conflict -- for no good reason. I also get file timestamp updates after save when I didn't do anything those objects. This makes Tortise see them as modified. Trivial changes, like saving on a different sheet, cause an update to the top level files.

3. A merge can get be very difficult. For example, if you and I make changes to the same chart, it is in my experience, very difficult to get a merge right -- or avoid destroying the chart.

So I've had good experience with Source Control for single user development or very low volume (follow up maint) multi-user (really serial - developer at a time). I actually find it easier to just check in reduced qvws in this scenario.

For high volume concurrent development, here's the approach I like. 

1. The script goes into it's own file, like Sales_DM.qvw.

2. The user facing qvw "Sales.qvw", binary loads "Sales_DM.qvw".

3. I'm the team leader, I work in the "master" copy Sales.qvw.

4. You make a copy of Sales.qvw and work on your assigned sheets/objects.

5. I Ctrl-c / Ctrl-v copy your objects to the master when you are ready and do the final positioning.

If anybody needs to make a change to the data model, they update Sales_DM and everyone picks up the change.

Optionally, the DM and the master can be checked in to create releases or history.

-Rob

http://masterssummit.com

http://robwunderlich.com

[stevelord]

Thanks Rob, the recommendations you provided were excellent given the complexity of the situation.  No magic wands or silver bullets, but definitely makes things less risky.

PS> I have scrutinized the prj folders and xml files, and actually did a test to see what-all changes if I just check a box in a table and check that in.  The chart's xml file changed like expected, and the other two files you mentioned, but the merge tool's display of the mine/theirs xml stuff didn't fit any logic I could think of for the act of checking a box.  I saw the last update date change which was fine, but seeing half a dozen trues go to false and no discernible reference to the object I changed in the merge tool's windows had me scratching my head.  I'm mostly solo on my side, but the team leader in a multi-developer scenario would likely be just as baffled unless they were a robot.  Team leader copy/pasting objects to the master qvw and checking that in is probably the best/easiest way for the team leader to know what happened, and be at ease just saying yes to all when checking in changes on the merge tool.

[stevelord]

Thanks Miguel for the must read white paper and learning ideas were helpful.  I had to give 'correct answer' to Rob for the practical applications aspect of his recommendations (though he did point back to the white paper and your post.  He outlined how a multi-developer team would function on top. )

[rwunderlich]

Thanks for the feedback on your test Steve. It parallels my experience.

-Rob