Alternative title: How to Make Your Life Easier and Not Go Bald from Pulling Your Hair out Trying to Keep Current on a Constantly Evolving and Shifting Codebase

This is probably the biggest and most avoided question from all of our successfully launched Professional Services projects. How can we upgrade to the greatest and latest on our heavily customized EP project? Let's be clear that performing a point release upgrade, or even consuming the latest sprint of code from our extremely productive Product Development group isn't trivial. A full version upgrade is even more daunting and usually heavily balanced against the cost of re-implementing from scratch.

First off, hopefully you are following our advice to try to avoid customizing EP code as far as possible, as noted in our documentation and tutorials. Following this methodology will make your life a lot easier, as upgrading core EP classes that have only minor changes will be a lot easier.

With that in mind, the best method through tried and tested experience, from brute force patching and merging, to re-implementing large customizations, is to take advantage of the merge tools from your chosen source control software (you are using source control I hope! If not, this book will be your best friend). Why do all the manual merge work when X does it so well already (X being subversion, perforce, cvs, vss, git, the next big flavor of the month). The key to leveraging your source control software to facilitate an upgrade is the concept of a vendor branch.

Working with a vendor branch is straight forward. This is applicable to any 3rd-party library you are using and modifying, not just EP. Take a quick gander at these various interweb sites for a better explanation than i can reasonably provide. For our purposes, a vendor branch for EP is an untainted, pristine copy of EP source. Don't just take our beloved zip of source code and stash it in your network drive! Check it in and show it off! You can branch your customized codeline off the vendor branch and start working away with the enthusiasm of a frenzied squirrel in the spring. The next time a new version of EP comes along, simply drop it directly on top of the current vendor branch of EP in your repository. With this new EP revision, you can perform your normal merge routine of just the diff between these EP revisions and merge the delta of changes into your customized eCommerce solution. And, if you have been following the procedures to avoid modifying EP source, conflicts should be kept to a minimum, files will just resolve themselves, and you can go out for a quick pint. Sure sounds easy huh? But how about in real life?

Well, let's see how we've taken this and made our lives easier at a client site in Perforce. I'm not the biggest Perforce fan, but I sure appreciate it's advanced branching and visual merging capabilities. We've set this up as follows:

• EP_IMPORT, our EP vendor branch containing 100% pure EP 6.1 eCommerce goodness
• Stable - our main development branch, full of customizations of the greatest and latest variety

Now, presume we're taking in change after massive change from our Product Development group. They've given us EP 6.1.1! Hooray! But how do we get this into our source control? Looks easy enough. Drop it on top of EP_IMPORT and check it in? Whoa, hold on there, code cowboy. This baby probably won't compile just with that.

The main things to remember is to collect three things with each new drop of code:

1. All files that have been updated in 6.1.1
2. All files that have been deleted in 6.1.1
3. All files that have been added in 6.1.1

Once you have all three, you can mark the relevant files for delete, add and update. I've taken a short cut to calculate the files that fall into the above three buckets in this case. I have access to our internal subversion repository at EP, so I'm just going to take a log of files taken in by 'svn update' when updating from the 6.1 tag and the new 6.1.1 tag. You can essentially calculate the same bucket of files using the diff command. But in my case I'm just taking a shortcut and doing:

svn update > merge.txt

This logs out for us a lot of lines of this variety:

U    com.elasticpath.core/WEB-INF/src/main/java/com/elasticpath/service/rules/PromotionRuleDelegate.java
D    com.elasticpath.core/WEB-INF/src/main/java/com/elasticpath/service/misc/impl/WorkAroundOpenJPAFetchPlanHelperImpl.java
A    com.elasticpath.core/WEB-INF/src/main/java/com/elasticpath/service/misc/impl/OpenJPAEventListeningFetchPlanHelperImpl.java

With some help from sed (for the non-Unixers, feel free to grab and install cygwin, or your favorite port of the beloved Unix command line tools), I'm going to parse the log and split it into three files:

cat merge.txt | sed \-e '/D /d' | sed \-e '/U /d' | sed \-e 's/A[ ]*//' > adds.txt
cat merge.txt | sed \-e '/A /d' | sed \-e '/U /d' | sed \-e 's/D[ ]*//' > deletes.txt
cat merge.txt | sed \-e '/A /d' | sed \-e '/D /d' | sed \-e 's/[ ]*[U]*[ ]*//' > updates.txt

Now, Perforce is a bit finicky in that I need to mark which files I'm going to edit to remove the read-only permissions of relevant files. Sure, I can mark every file for editing, but let's play it safe:

pwd
/perforce/depot/branches/EP_IMPORT/
cat updates.txt | xargs p4 edit -f

Next, time to mark the deleted files for removal:

cat deletes.txt | xargs p4 delete -f

Finally, it's time to unzip on top all the files from the new EP 6.1.1 source zip and mark our new files for add:

tar -xvzf EP6.1.1_Export.tgz .
cat adds.txt | xargs p4 add -f

That does it! Now check that all in! Now we have EP 6.1.1 in our EP_IMPORT branch. Time to use the impressive P4V tool from Perforce to integrate from the EP_IMPORT branch into the Stable branch. Since we've avoided modify EP source as much as possible, the majority of the changes should auto-resolve. Clean up some of the conflicts and tweak the customizations as necessary, run the normal routine of compile, checkstyle, pmd, unit tests, integration tests and your project should now be ready to go. Okay, that's a gross over-simplication of the conflict resolution and regression testing process that goes hand in hand with a version upgrade, but hopefully you see how a vendor branch does wonders in facilitating an easier upgrade process with custom code. No more awkward patch files and brute force.

Assuming you don't have access to our subversion to get a hold of the list of changes, you can use diff to log file differences and files missing from the 6.1 and 6.1.1 codebases.

diff -r /perforce/depot/branches/EP_IMPORT /tmp/EP611/ > diffs.txt

This should give you a nice log that is easily parsed into the same add/delete/update buckets. So now you're freshly armed to do the upgrade that has been haunting you late at night. Vendor branches and merge tools are your new best friends! Happy upgrading and merging.

Drew