git-svn, meet git-p4

Many developers have started to use Git, a distributed VCS that offers a lot of advantages over centralized repositories when it comes to team development.  This is further fueled by the success of the integration tool between Git and Subversion, git-svn.  But did you know Git can also interact with Perforce?  While not as mature as its Subversion counterpart, git-p4 has the basic feature set to achieve the same objective: the ability to leverage Git for your daily work and only connect to Perforce when necessary.

The scenario

You're working on a project trunk for a customer, whose organization uses Perforce.   The customer has given you the proper Perforce access and expects your team to be committing regularly against it.  On the other hand, you need to maintain this project in your organization's Subversion as well.

So, you start off by doing an initial code drop to Perforce.  But what's the best way to bring your changes over?  The most obvious approach is doing a manual merge at regular intervals.  If done frequently and carefully, this may be the way to go.  However, besides the obvious disadvantages  (time consuming, human error), we also lose our original commit history.  What we would really like is a way to forward-port our commits from Subversion to Perforce.  Let's see how we can leverage Git to do this for us.

A thousand words...

There's a lot happening in this diagram, but the basic flow is easy to understand.

First, we set things up by cloning the project from Subversion and Perforce.  This is represented by Git branches master and master_perforce, respectively.

Meanwhile, the team hacks away and eventually, commit E is committed to Subversion.  The person performing the "merge" (we're not really merging here, it's a rebase as we will see later) ensures his/her repository is fully synced with Subversion (git svn rebase or git svn dcommit).

Next, what follows is the repeatable workflow to be done at regular intervals:

1. Identify commit(s) to be ported to master_perforce.  In this case that would be commit E.
2. Forward-port commit E from master to master_perforce, which then creates a new commit E' on master_perforce.
3. Submit commit E' to the remote Perforce repository.

The rest of this article will now explain in detail the setup step as well as the repeatable workflow.

Cloning the project from Subversion and Perforce

This is one-time setup.  In fact, if you are already using git svn for your project, you can even skip the step on cloning the Subversion project.

Also note that the commands are based on Mac OS X with Bash.

Download and install the tools

1. Download git at http://git-scm.com/.  Make sure to download 1.7.1 or later.
2. Download git-p4 at https://github.com/ermshiperete/git-p4.  As Perforce recommends, do not use any other version other than the one provided by this link.
   • So we can invoke it from command line like git p4, we need to create a soft link.
     

sudo ln -s ~/apps/git-p4 /usr/local/git/libexec/git-core

3. git-p4 is a Python script, so download Python if necessary.
4. Download p4 at http://www.perforce.com/product/components/perforce_commandline_client and add it to your path.

   sudo ln ~/apps/p4 /usr/bin/p4

Configure Perforce client settings

Create a properties file, say settings.config.

P4PORT=<Perforce server URL>:<Perforce server port>
P4USER=<username>
P4PASSWD=<password>
P4EDITOR=<path to text editor, e.g. "C:\Windows\notepad.exe" or vim>
P4CLIENT=projectx_trunk_ws

The last variable, P4CLIENT denotes the workspace you will be creating in the next step, so feel free to improvise.

Create environment variable P4CONFIG and point to the properties file.

export P4CONFIG=~/settings.config

Create Perforce client workspace

We need to create the Perforce client workspace.  Perforce requires a client workspace that defines a "client view".  This view determines the the path that the workspace is mapped to in Perforce.  For this article, we'll make the assumption that the path of the initial code drop to Perforce is //depot/projectx/trunk.  You'll also need to create a directory for this client workspace, I chose the same name but it doesn't have to be.

mkdir projectx_trunk_ws
cd projectx_trunk_ws
p4 client projectx_trunk_ws

The last line should launch the editor specified in P4EDITOR.  Look for the View: field which has the format [path in Perforce] maps to [path on your machine].  So, change //depot/... to //depot/projectx/trunk.

View:
        //depot/projectx/trunk/... //projectx_trunk_ws/...

Save and exit.

Test Perforce connectivity

p4 info should give Perforce server connectivity as opposed to an error. So try it out now!

Clone the Subversion project

This should be famiar to those already using git-svn to connect to Subversion.

mkdir projectx.git
cd projectx.git
git svn clone http://subversion/projectx/trunk .

Add your standard Git configuration, etc...

git config user.email "youremail@elasticpath.com"
git config user.name "username"
...

git branch should now show that we have master.  Not surprisingly, this represents the trunk in Subversion.

rlim@project.git> git branch
* master

Clone the Perforce project

cd projectx.git
git p4 sync //depot/projectx/trunk@all

@all will bring in all history. If you just want the head revision, leave that out.

Unlike git svn clone, the command git p4 sync does not automatically create a development branch for us.  This is where we create perforce_master, which, not surprisingly, represents the trunk in Perforce.

git branch master_perforce remotes/p4/trunk

Where did remotes/p4/trunk come from?  Similar to git svn clone, the command git p4 sync creates a remote (tracking) branch named remotes/p4/trunk to track changes made in the Perforce trunk.  You don't do work on this branch, you need to check out this branch to a development (topic) branch, master_perforce.  Although not shown, git svn clone also created a remote branch behind the scenes.  The difference is it automatically created a development branch master off this remote branch for us.

Where are we now?

That's a whole lot of set up, but thankfully it is just one-time.  git branch should now show two branches.

 ______________________
rlim@projectx.git> git branch
  master
* master_perforce

Identify the commits

This signals the start of the repeatable workflow as described previously.  In this step we will need to identify the "candidate" commits - those that we want to port over to master_perforce.  This boils down to determining the start and end point for porting.  Any commits between these "points" will be ported over.

Remember, you've done initial code to Perforce, which means at some point in Subversion history, the code was copied to Perforce.  Since we've cloned the history from Subversion, we have the commit history available in Git.  This needs to be figured out in relation to the most current commit, HEAD.  For our example, let's assume this was done at HEAD^.  Couple of ways to look at it just to drill the point home:

• It was one revision ago (HEAD^) that we dropped the code to Perforce, or ...
• ... since we dropped the code to Perforce, we've made one commit in Subversion - that's commit E.

In Git, a symbolic reference is a name that points to an object.  HEAD is such a symbolic reference and the object that it points to is the most recent commit of the branch you are currently in.  Because a parent commit is always reachable from any  commit, we can refer to previous commits using the notation HEAD^ (previous commit from most recent) or HEAD~1 (previous commit again) or HEAD~2 (two commits back from most recent).  Much easier than specifying the commit object's SHA-1 hash!

So, we now know when we did the initial code drop, and it also means commit after this we'll need to port over to master_perforce.  The initial code drop becomes the starting point for porting.  Let's tag this initial starting point.

git checkout master
git tag initial_code_drop HEAD~1

What about the ending point?  Assuming we want all commits ported over, the end point is already defined for us - that's HEAD.  Putting it together, we want all the commits between HEAD~1 (excluding) and HEAD (including).  In our simple, hypothetical scenario, that's just one commit, E

All Your Rebase Belong to ... Git?!

We have identified the starting and end point in which to port our commits over from master to master_perforce.  Here's the actual commands to do the actual forward-porting.  The lines are numbered for further analysis.

1   git checkout master
2   git branch -f start initial_code_drop
3   git branch -f end HEAD
4   git rebase --onto master_perforce start end
5   git branch -f master_perforce end

1 - Checkout master branch.

2 - Rebase works off branches.  So, we must create branch start (denoting our starting point) off the tag initial_code_drop.

3 - Same reason as previous, except for branch end (denoting our ending point) off HEAD.

4 - This command reads, "forward port all commits onto master_perforce after start (remember, it's excluding) and ending at end".

5 - After the previous command is executed, master_perforce has not changed but end has been rebased onto it.  We need to re-point master_perforce to end (also known as a fast-forward merge of branches master_perforce and end).  For those uninitiated to rebasing and branching with Git, search for "rebase" on the Wiki and hopefully my rambling there will shed more light on this.

Line 4 is the one that does the heavy lifting.  Git takes each commit and applies it (as a patch) sequentially to master_perforce.  We also have two branches, start and end that are no longer needed.  It's safe to delete these branches (e.g. git branch -d start), but you don't really have to, since they will be used again the next time around.

Tag new starting point for next round

As the title states, we should tag where we last left off.

git tag 09Dec HEAD

This will become the new starting point (instead of initial_code_drop) for the next round of rebasing!  By using the current date, I know when was the last time I performed this workflow (to see the actual commit that was tagged, i.e. HEAD at that time, git show 09Dec).

If you look at the 5 lines above again, it means everytime we do this, the only line that changes is line 2!  Better yet, dump these lines to a script that take a single argument - the starting point.

Committing to Perforce

Hopefully this should be the easiest step of the workflow.  Review your changes, as this is the point of no return!  If you want to revert before the rebase, the commands git checkout master_perforce and then git reset --hard HEAD~1 will undo commit E'.

git checkout master_perforce
git-p4 submit

Each commit will bring up the editor specified in P4EDITOR.  Notice the commit message is already populated with the exact message that was committed in Subversion for commit E.

Final thoughts

Porting a commit that was previously ported

If you lose track of the starting point, and accidentally port over a commit that was previously ported, don't fret!  The rebase operation works on patches and Git is smart enough to know it was already applied previously.  For this commit you will get a message like No changes – Patch already applied.  Git will happily continue along applying the other commits sequentially.

Cherry picking a range of commits

As of Git 1.7.2, there is another approach to port commits over instead of using rebase.  You can specify a range of commits with git cherry-pick start..end, which would essentially be a one liner operation instead of five lines above.  However, I have yet to try this and secondly, I don't believe cherry picking offers the chance to continue after a merge conflict as rebase does.