The MouseFeed update site is located at: http://update.mousefeed.com/

One of the pain points that can arise during a development project is the payment gateway's support of recycled order numbers. Live production gateways will track all historical order numbers and generate a duplicate order number error if an existing order number is reused. This is fine in production but troublesome when the payment gateway test servers also behave in this manner. Most payment gateway providers I have worked with in the past offer duplicate checks within an hour or two on their test servers, but beyond that, order numbers can be recycled which is a good thing for development purposes. Some providers, however, appear to extend this duplicate check to a day, week, or may even provide no support for order number reuse.

If you use the database population scripts that come with Elastic Path to facilitate schema/data changes, this limitation can frustrate your development team, who will most likely end up manually tracking used order numbers and making frequent updates to TORDERNUMBERGENERATOR.

If your payment gateway provider doesn't support order number reuse or at least not to the frequency you desire, what can you do?

Some Alternatives

The simplest approach would be to externalize the initial order number to the env.config and assign everyone a unique order number block. The problem with this is that each environment would have to keep track of the last order number used and update the env.config with the next number in the block with every new database build.

Another approach would be to externalize the initial order number (or perhaps even the order number incrementer) to a central database. This database would have a simple mapping of environment to next order number and each environment would have a sufficient block assigned to last the duration of the project. The problem with this approach is that many teams/environments are distributed and you may not be able to set up a central database which can be accessed by all environments.

A Better Solution

A better solution (though not 100% foolproof) is to construct the initial order number during the database build process using a unique element of the environment as well as a random element to get around frequent rebuilds on the same environment. To do this, we can create a custom Ant task that takes a fixed component (derived from either a fixed prefix or the environment's IP address) and appends a randomly generated number. If the random number is sufficiently large, the likelihood of duplicate hits will be minimal.

Creating the Ant Task

The following is an example of a custom Ant task that accepts min and max parameters for the random number component and a prefix. Regardless of whether a fixed prefix or an environment's IP address was provided, the task will use the hash code of the fixed value for the resulting order number in order to work with the out of the box numerical order number incrementer.

public class OrderNumberGeneratorTask  extends Task {
    private String min = null;
    private String max = null;
    private String property = null;
    private String prefix = null;
    private Random random = new Random(System.currentTimeMillis());
 
    @Override
    public void execute() throws BuildException {
         String localPrefix = null;
        if (min == null || min.equals(""))
            throw new BuildException("Min property is missing.");
 
        if (max == null || max.equals(""))
            throw new BuildException("Max property is missing.");
 
        int minInt = Integer.parseInt(min);
        int maxInt = Integer.parseInt(max);
 
        if (minInt > maxInt)
            throw new BuildException("Min is bigger than max.");
 
        localPrefix = prefix;
        if (localPrefix == null) {
             try {
                  localPrefix = InetAddress.getLocalHost().getHostAddress();
               } catch (UnknownHostException e) {
                    throw new BuildException("Unable to get local host address.", e);
               }
        }
        int randomInt = calculateRandom(minInt, maxInt);
 
        getProject().setNewProperty(property, String.valueOf(localPrefix.hashCode()) + String.valueOf(randomInt));
    }
 
    protected int calculateRandom(final int minInt, final int maxInt) {
        return minInt + random.nextInt(maxInt - minInt + 1);
    }

Updating the Build Process

To use this task, we need to package the compiled class in a jar and add it to the Maven repository along with an entry in the ant/setup/pom.xml so that it gets included in the ant setup script. Once the class is available on the Ant class path, we add the following task definition and task invocation to the ant/default.xml setup file. This will create the order number and store it in the ep.initial.order.number variable.

<?xml version="1.0" encoding="UTF-8"?>

<project name="ant_default" xmlns:artifact="antlib:org.apache.maven.artifact.ant" xmlns:contrib="antlib:net.sf.antcontrib">

  <taskdef name="ordernumbergenerator" classname="com.elasticpath.antextension.OrderNumberGeneratorTask"></taskdef>
... 
  <ordernumbergenerator min="1" max="1000000" property="ep.initial.order.number"></ordernumbergenerator>
...

Now we can use this variable in the database/src/base-insert.xml.vm file as the initial order number of the TORDERNUMBERGENERATOR table.

     <Tordernumbergenerator Uidpk="1" NextOrderNumber="${ep_initial_order_number}" />

(Or: How to Copy and Paste XML Your Way to Greatness)

We have a truckload of Selenium tests that poke and probe away at the user interface of our project here at EP. We realized it would be awesome if this testing could just automagically happen and let us know the results. So we took a look at our Maven build and realized the steps we wanted to automate were:

1. Check out our source code, build the latest set of artifacts and deploy them to our internal Maven repository.
2. Stop the web application running on our QA server environment.
3. Recreate our test MySQL database with our default schema, apply any change sets needed to bring it to the latest version and load our QA data.
4. Update the test data to reflect our QA server environment.
5. Deploy our demo assets to a known location.
6. Start our web application and wait for it to fully start.
7. Launch Selenium and run our tests against our application on Firefox on our Linux Hudson slave.
8. Record and announce the results of the tests.

In this blog post, I'm going to cover how we automated the last three steps.

The first thing we needed to do was find a tool that could automate configuring a web app container and deploying our WAR files. This tool would need to support multiple web containers without having to write custom scripts for each one. After doing some research, we decided to go with the fairly capable Cargo project. It has a fairly mature Maven plugin and good support for our preferred container, Tomcat, and it's come a long way since its inception. On our side, all we'd need to do is write a simple Groovy script to wait until our web application is in a ready-to-test state.

Our particular application really benefits from the new WebDriver features in Selenium 2.0, and while we'd love to take advantage of the Selenium Plugin for Maven, it's tied to Selenium 1.0, so we can't fully use it yet. We can however use it to bootstrap an X server on our Hudson build slave.

Our Selenium tests are written using TestNG and stored in their own Maven artifact. We can easily bind the Maven Failsafe Plugin to run during our integration-test phase, but we'd like to publish the output to our internal reporting site, so we can take a look at the last results. Luckily, we can use the Maven Surefire Report plugin to generate our site. Lastly, we'll get Hudson to announce the results of our test. (The page on the Maven build lifecycle is invaluable when figuring out when to run different plugins.)

So, let's get started!

Properties

To make Cargo and Selenium more useful, we pushed a handful of settings into a parent POM. This lets us override values and use the same artifacts for our own local development testing.

<properties>
    <cargo.jvmargs>-XX:MaxPermSize=512M -Xmx1024m</cargo.jvmargs>
    <cargo.servlet.port>8080</cargo.servlet.port>
    <cargo.tomcat.ajp.port>8009</cargo.tomcat.ajp.port>
    <cargo.rmi.port>1099</cargo.rmi.port>
    <cargo.tomcat.shutdown.port>8205</cargo.tomcat.shutdown.port>
    <cargo.wait>true</cargo.wait>

    <demo.hostname>10.9.8.7</demo.hostname>

    <searchserver.context>searchserver</searchserver.context>
    <ourapp.context>ourapp</ourapp.context>
    <cmserver.context>cmserver</cmserver.context>
    <storefront.context>storefront</storefront.context>

    <!-- Absolute -->
    <assets.directory>${user.home}/ep-assets</assets.directory>

    <jdbc.driver>com.mysql.jdbc.Driver</jdbc.driver>
    <jdbc.host>10.9.8.7:3306</jdbc.host>
    <jdbc.db>OURAPP_DB</jdbc.db>
    <jdbc.username>atwill</jdbc.username>
    <jdbc.password>grep</jdbc.password>
</properties>

Okay, that's more than a handful. but let's go over them really quickly. The cargo.* properties are there to let us easily have multiple Tomcats running on the same IP, the *.context properties tell Cargo where to deploy our various WAR files. The cargo.wait property tells Cargo to pause after it starts the application server. If set to false, Cargo will proceed along the Maven build lifecycle.  The assets.directory is used by steps 4 and 5.  The demo.hostname is the public hostname (or IP in our case) for our deployment (you'll see why we do this as we move along). The jdbc.* properties describe our JDBC DataSource. We've broken the information up so that we can use it in steps 3, 4 and 6. As you can imagine, it's pretty easy to override these properties in your local settings.xml and use Cargo to start a local app server for you - lucky you!

Containing the Excitement

Our first goal is to start a container, deploy our artifacts and wait for them to all start. For our situation, we felt it was best to have a separate container Maven artifact (with pom packaging) dedicated to steps 5 and 6. So, first we'll declare the WAR and JAR files that we depend on for the container. We included the JDBC driver and the JAI JARs for our Storefront too. We've already declared the versions for all these artifacts in a parent POM.

....            
           <dependencies> 
                <!-- WARs to be deployed by Cargo (also add them in cargo below) -->
                <dependency>
                        <groupId>com.elasticpath.mdm</groupId>
                        <artifactId>ourapp-war</artifactId>
                        <type>war</type>
                </dependency>

                <dependency>
                        <groupId>com.elasticpath.mdm</groupId>
                        <artifactId>searchserver</artifactId>
                        <type>war</type>
                </dependency>

                <dependency>
                        <groupId>com.elasticpath.mdm</groupId>
                        <artifactId>cmserver</artifactId>
                        <type>war</type>
                </dependency>

                <dependency>
                        <groupId>com.elasticpath.mdm</groupId>
                        <artifactId>storefront</artifactId>
                        <type>war</type>
                </dependency>

                <!-- Dependencies provided to Cargo -->
                <dependency>
                        <groupId>mysql</groupId>
                        <artifactId>mysql-connector-java</artifactId>
                        <scope>provided</scope>
                </dependency>

                <dependency>
                        <groupId>javax.media</groupId>
                        <artifactId>jai-core</artifactId>
                        <scope>provided</scope>
                </dependency>

                <dependency>
                        <groupId>com.sun.media</groupId>
                        <artifactId>jai-codec</artifactId>
                        <scope>provided</scope>
                </dependency>
        </dependencies>

Next up, we'll start Cargo in the pre-integration-test phase and stop it when we hit the post-integration-test phase. If you're copying and pasting this XML, you'll want to paste the next three sections in sequence into your POM, but we'll cover them individually.

<build>
    <plugins>
        <plugin>
            <groupId>org.codehaus.cargo</groupId>
            <artifactId>cargo-maven2-plugin</artifactId>
            <version>1.0.1-beta-2</version>
            <executions>
                    <execution>
                        <id>start-container</id>
                        <phase>pre-integration-test</phase>
                        <goals>
                            <goal>start</goal>
                        </goals>
                    </execution>
                    <execution>
                        <id>stop-container</id>
                        <phase>post-integration-test</phase>
                        <goals>
                            <goal>stop</goal>
                        </goals>
                    </execution>
            </executions>

            <configuration>
                <container>
                    <dependencies>
                        <dependency>
                            <groupId>mysql</groupId>
                            <artifactId>mysql-connector-java</artifactId>
                        </dependency>

                        <dependency>
                            <groupId>javax.media</groupId>
                            <artifactId>jai-core</artifactId>
                        </dependency>

                        <dependency>
                            <groupId>com.sun.media</groupId>
                            <artifactId>jai-codec</artifactId>
                        </dependency>
                    </dependencies>

                    <containerId>tomcat5x</containerId>
                    <zipUrlInstaller>
                        <url>file:${project.basedir}/apache-tomcat-5.5.29.zip</url>
                        <installDir>${installDir}</installDir>
                    </zipUrlInstaller>
                </container>

In the previous example, we bind the plugin to the lifecycle and tell Cargo to include the JDBC driver and JAI JARs into the classpath. (We won't get native acceleration for JAI, but that's fine for our purposes.) You'll probably notice the tricky thing we did for the zipUrlInstaller. To remove a dependency from our startup, we've checked in a copy of Tomcat into our project. Let's take a look at the Cargo configuration:

<configuration>
    <home>${project.build.directory}/tomcat5x/container</home>
    <properties>
        <cargo.jvmargs>${cargo.jvmargs}</cargo.jvmargs>
        <cargo.logging>high</cargo.logging>
        <cargo.servlet.port>${cargo.servlet.port}</cargo.servlet.port>
        <cargo.tomcat.ajp.port>${cargo.tomcat.ajp.port}</cargo.tomcat.ajp.port>
        <cargo.rmi.port>${cargo.rmi.port}</cargo.rmi.port>
        <cargo.tomcat.shutdown.port>${cargo.tomcat.shutdown.port}</cargo.tomcat.shutdown.port>
        <!--
            Ampersands must be escaped such that they show up as "&" in
            the resulting XML file, so we use "&amp;" here.
        -->
        <cargo.datasource.datasource>
            cargo.datasource.url=jdbc:mysql://${jdbc.host}/${jdbc.db}?AutoReconnect=true&amp;amp;useUnicode=true&amp;amp;characterEncoding=utf-8|
            cargo.datasource.driver=${jdbc.driver}|
            cargo.datasource.username=${jdbc.username}|
            cargo.datasource.password=${jdbc.password}|
            cargo.datasource.type=javax.sql.DataSource|
            cargo.datasource.jndi=jdbc/epjndi
        </cargo.datasource.datasource>
    </properties>

Here we're passing in the cargo.* properties from our parent POM and we're asking Cargo to create a DataSource for us with the jdbc.* properties and add it to the JNDI tree.  For our situation, we're always using MySQL, so we can make a handful of assumptions in the URL.  Now that we've picked a container and configured it, we'll tell Cargo which artifacts we'd like to have deployed:

            <deployables>
                <!-- Applications to Deploy -->
                <deployable>
                    <groupId>com.elasticpath.mdm</groupId>
                    <artifactId>ourapp-war</artifactId>
                    <type>war</type>
                    <properties>
                            <context>/${ourapp.context}</context>
                    </properties>
                    <pingURL>http://${demo.hostname}:${cargo.servlet.port}/${ourapp.context}
                    </pingURL>
                    <pingTimeout>60000</pingTimeout>
                </deployable>
                <deployable>
                    <groupId>com.elasticpath.mdm</groupId>
                    <artifactId>searchserver</artifactId>
                    <type>war</type>
                    <properties>
                            <context>/${searchserver.context}</context>
                    </properties>
                    <pingURL>http://${demo.hostname}:${cargo.servlet.port}/${searchserver.context}/product/select?q=*:*
                    </pingURL>
                    <pingTimeout>60000</pingTimeout>
                </deployable>
                <deployable>
                    <groupId>com.elasticpath.mdm</groupId>
                    <artifactId>cmserver</artifactId>
                    <type>war</type>
                    <properties>
                            <context>/${cmserver.context}</context>
                    </properties>
                    <pingURL>http://${demo.hostname}:${cargo.servlet.port}/${cmserver.context}/
                    </pingURL>
                    <pingTimeout>60000</pingTimeout>
                </deployable>
                <deployable>
                    <groupId>com.elasticpath.mdm</groupId>
                    <artifactId>storefront</artifactId>
                    <type>war</type>
                    <properties>
                            <context>/${storefront.context}</context>
                    </properties>
                    <pingURL>http://${demo.hostname}:${cargo.servlet.port}/${storefront.context}/
                    </pingURL>
                    <pingTimeout>60000</pingTimeout>
                </deployable>
            </deployables>
        </configuration>
    </configuration>
</plugin>

This is pretty much textbook Cargo; you'll see our use of demo.hostname above in PingURL. Cargo will fetch the artifacts and deploy them into Tomcat waiting for the specified URL to not return an error, waiting up to pingTimeout milliseconds. Since we're just a demo application, the PingURL for the searchserver does a query for products. The problem here is that, while the searchserver may be available, the search indexes might not be built yet. So, we'll need something to check that the searchserver is not only started, but that its indexes are populated.

For this, we put together a simple Groovy script that waits until the searchserver returns products when queried. We put this in src/main/script/waitforstartup.groovy and it looks a little something like this:

import java.net.URL
import groovy.xml.StreamingMarkupBuilder


 def url = project.properties['searchserverUrl'];
 def timeout = project.properties['timeout'].toLong();

 /* For standalone testing, use something like:
  * 
  *     def url = "http://localhost:8080/searchserver/product/select?q=*:*"
  *     def timeout = 120*1000;
  */

def ready = false;
def text = "";
def first = true;

def begin = System.currentTimeMillis()

while (!ready) {
        
        if (System.currentTimeMillis() > (begin+timeout)) {
                // provided by gmaven
                fail("Timeout of "+timeout+"ms reached waiting for "+url+" to return at least one search result.")
        }
        
        if (!first) {
                Thread.sleep(10*1000);
        } else {
                first = false;
        }
        
        URL searchServer;
        try     {
                searchServer = new URL(url);
                text = searchServer.getText();
        } catch (e) {
                print "Server not yet ready, sleeping for 10 seconds. ("+e+")\n"
                continue;
        }
        
        if (text.size() == 0) {
                print "Not yet ready, sleeping for 10 seconds. (Empty HTTP response received)\n"
                continue;
        }
        
        try {
                def root = new XmlSlurper().parseText(text)
                
                if (numFound(root)!=0) {
                        ready = true;
                } else {
                        print "No elements returned in search result, sleeping for 10 seconds.\n"
                        continue;
                }
        } catch (e) {
                print "Trouble parsing XML result, sleeping for 10 seconds. ("+e+")\n"
                continue;
        }
}

def numFound(root) {
        return root.result.@numFound;
}

XmlSlurper was pretty neat (as used in numFound). Running it is pretty trivial with the GMaven plugin. You'll notice how we can easily pass in properties to the Groovy script via project.properties.

<plugin> 
    <groupId>org.codehaus.groovy.maven</groupId>
    <artifactId>gmaven-plugin</artifactId>
    <configuration>
        <properties>
            <timeout>300000</timeout>
            <searchserverUrl>http://${demo.hostname}:${cargo.servlet.port}/${searchserver.context}/product/select?q=*:*</searchserverUrl>
        </properties>
        <source>${basedir}/src/main/script/waitforstartup.groovy</source>
    </configuration>
</plugin>

The source element is relative to the current working directory, not your project's base directory, so it's a good idea to put in ${basedir} to prevent surprises later on.

Running mvn integration-test should scroll for a while as Maven downloads all required dependencies and eventually the container will start. If we run gmaven:execute, our Groovy script will run and happily poll our searchserver until it's ready!

This alone is a great set up for running builds of our project both on my development box and for our QA server!

In Hudson, I have one Job which spawns both targets in parallel.  The integration-test job never returns (until it's killed in step 2), but once the gmaven:execute target returns, it runs the Selenium tests.  Let's talk about those now...

Prepare for Take Off

Our second goal is to launch Selenium, run our tests and shut Selenium down.  It made sense for us to have a separate selenium-tests artifact for that purpose, let's start by taking a look at the properties we decided to set:

<properties>
    <selenium.version>2.0a4</selenium.version>
    <selenium.host>127.0.0.1</selenium.host>
    <selenium.port>14444</selenium.port>
    <selenium.DISPLAY>:17</selenium.DISPLAY>
    <selenium.background>true</selenium.background>
    <xvfb.option>-ac</xvfb.option>
    <seleniumUrl>http://${selenium.host}:${selenium.port}/wd/hub</seleniumUrl>
    <appUrl>http://${demo.hostname}:${cargo.servlet.port}/${ourapp.context}</appUrl>
</properties>

We're making use of the Maven Selenium Plugin to start up Xvfb on display :17, and we disable access control to Xvfb to work around any xauth issues we may run into.  Here you see another use of demo.hostname, passing it to Selenium as the application under test.  The selenium.background property allows us to run Selenium on the command line and keep the server running with -Dselenium.background=false, this running these tests during development makes testing a lot quicker!

Next we'll specify the dependencies we need to make this all happen:

<dependencies>
    <dependency>
        <groupId>org.seleniumhq.selenium</groupId>
        <artifactId>selenium</artifactId>
        <version>${selenium.version}</version>
    </dependency>

    <dependency>
        <groupId>org.testng</groupId>
        <artifactId>testng</artifactId>
        <version>5.11</version>
        <scope>test</scope>
        <classifier>jdk15</classifier>
    </dependency>

    <dependency>
        <groupId>org.seleniumhq.selenium</groupId>
        <artifactId>selenium-server</artifactId>
        <version>${selenium.version}</version>
    </dependency>
</dependencies>   
<repositories>
    <repository>
        <id>selenium-repository</id>
        <url>http://selenium.googlecode.com/svn/repository/</url>
    </repository>
</repositories>

No real surprises here, although I'm adding an extra repository just to pull down Selenium 2.0a4. You could easily mirror it locally in your own company-wide repository.

Now we need to start and stop Selenium.  Normally we'd use the Maven Selenium Plugin, but it's pinned to the older generation of Selenium. Instead of trying to be overly clever, we just ask Ant to start and stop Selenium during the pre-integration-test and post-integration-test phases, we pass in a DISPLAY variable which is ignored on non-Unix operating systems.

<build>
    <plugins>
        <plugin>
            <!--
                Use Ant to start and stop the selenium server - once the selenium
                plugin handles 2.0 we can get rid of this.
            -->
            <groupId>org.apache.maven.plugins</groupId>
            <artifactId>maven-antrun-plugin</artifactId>
            <executions>
                <execution>
                    <id>start-selenium</id>
                    <phase>pre-integration-test</phase>
                    <configuration>
                        <tasks>
                            <echo taskname="start-selenium"
                                message="Starting Selenium Server v${selenium.version} on ${selenium.port} offering display ${selenium.DISPLAY}" />
                            <java taskname="start-selenium"
                                jar="lib/selenium-server-standalone-${selenium.version}.jar"
                                fork="true" spawn="${selenium.background}">
                                <env key="DISPLAY" value="${selenium.DISPLAY}" />
                                <arg line="-timeout 30 -debug -browserSideLog -port ${selenium.port}" />
                            </java>
                        </tasks>
                    </configuration>
                    <goals>
                        <goal>run</goal>
                    </goals>
                </execution>
                <execution>
                    <id>stop-selenium</id>
                    <phase>post-integration-test</phase>
                    <configuration>
                        <tasks>
                            <echo message="Stopping Selenium Server" />
                            <get taskname="stop-selenium"
                                src="http://${selenium.host}:${selenium.port}/selenium-server/driver/?cmd=shutDown"
                                dest="${project.build.directory}/selenium-shutdown.txt"
                                ignoreerrors="true" />
                        </tasks>
                    </configuration>
                    <goals>
                        <goal>run</goal>
                    </goals>
                </execution>
            </executions>
        </plugin>

Okay, so we've configured Selenium to start and stop, now lets ask it to run our tests. We created an extremely simple TestNG suite in src/test/it/testng.xml:

<suite name="integration-tests">
     <test name="Everybody Everybody">
          <packages>
               <package name="com.elasticpath.mdm.tests.it" />
          </packages>
     </test>
</suite>

By convention, the Maven Failsafe Plugin will run src/test/java/<above packages>/*IT.java tests when called with integration-test and TestNG will look for methods annotated with @Test.  So back in the POM, we say:

    <plugin>
        <!--
            Run tests specified in the testng.xml file for integration-tests
        -->
        <groupId>org.apache.maven.plugins</groupId>
        <artifactId>maven-failsafe-plugin</artifactId>
        <version>2.5</version>

        <configuration>
            <suiteXmlFiles>
                <suiteXmlFile>src/test/it/testng.xml</suiteXmlFile>
            </suiteXmlFiles>
            <systemPropertyVariables>
                <seleniumUrl>${seleniumUrl}</seleniumUrl>
                <appUrl>${appUrl}</appUrl>
            </systemPropertyVariables>
        </configuration>

        <executions>
            <execution>
                <id>integration-test</id>
                <phase>integration-test</phase>
                <goals>
                    <goal>integration-test</goal>
                </goals>
            </execution>
        </executions>
    </plugin>
</plugins>
 

We bind it to the integration-test phase so it's run after Selenium starts. You'll notice that we pass seleniumUrl and appUrl into the TestNG tests. We pass those as parameters into a @BeforeSuite method; we'll touch on that in a moment.

Last bit, we're going to launch Xvfb, but only if we're on a UNIXy operating system, so we'll wrap it in a profile stanza:

<profiles>
    <profile>
        <id>xvfb-started</id>
        <activation>
            <os>
                <family>unix</family>
            </os>
        </activation>
        <build>
            <plugins>
                <plugin>
                    <!--
                        Before running the Selenium server, start a Xvfb to display
                        browsers into.
                    -->
                    <groupId>org.codehaus.mojo</groupId>
                    <artifactId>selenium-maven-plugin</artifactId>
                    <executions>
                        <execution>
                            <id>xvfb</id>
                            <!--
                                We want to start Xvfb *before* pre-integration-tests, but the
                                only phase before pre-integration-test is package, so we put it
                                there.
                            -->
                            <phase>package</phase>
                            <goals>
                                <goal>xvfb</goal>
                            </goals>
                            <configuration>
                                <display>${selenium.DISPLAY}</display>
                                <options>
                                    <option>${xvfb.option}</option>
                                </options>
                            </configuration>
                        </execution>
                    </executions>
                </plugin>
            </plugins>
        </build>
    </profile>
</profiles>

Oh the hack.  Ironically, Maven does not have a way to specify that a particular execution id depends on another execution id, so we had to bind launching Xvfb to the package phase. Maybe in Maven 3.0?

Getting into the actual code, earlier I mentioned the @BeforeSuite attribute, in that method we'll configure Selenium for our purposes:

.....
 
        /*
         * When running TestNG from within Eclipse, the plugin passes this value in to parameters it cannot substitute.
         */
        private static final String PARAMETER_NOT_FOUND = "not-found";
.....
        /*
         * Passing in these properties will allow you to use the specified values if the parameters are not found by TestNG.
         */
        public static final String SELENIUM_URL_PROPERTY = "selenium.url";
 
        public static final String APPLICATION_URL_PROPERTY = "app.url";
 
        private static WebDriver driver;
 
        private static WebDriverBackedSelenium selenium;
 
        private static String sessionString;
        private static String appUrl;
.....
        @BeforeSuite
        @Parameters( { "seleniumUrl", "appUrl" })
        public void startBrowser(String seleniumUrl, String appUrl) throws Exception {
                
                if (PARAMETER_NOT_FOUND.equals(seleniumUrl)) {
                        seleniumUrl = System.getProperty(SELENIUM_URL_PROPERTY);
                }
                if (PARAMETER_NOT_FOUND.equals(appUrl)) {
                        appUrl = System.getProperty(APPLICATION_URL_PROPERTY);
                }
 
                driver = new RemoteWebDriver(new URL(seleniumUrl), DesiredCapabilities.firefox());
 
                // This will cause all find-element operations to keep trying for up to 10 seconds automatically.
                driver.manage().timeouts().implicitlyWait(10, TimeUnit.SECONDS);
 
                selenium = new WebDriverBackedSelenium(driver, appUrl);
 
                /*
                 * Generate a unique string so if we update a field to a new value, we know it's going to be a new value (and can compare that it's so).
                 */
                sessionString = System.currentTimeMillis() + "";
 
                selenium.open(appUrl);
 
        }
....
        @AfterSuite
        public void stopBrowser() {
                driver.close();
 
        }
.....
 

This method is called before our test suite runs.  We do some extra work to allow developers to run TestNG with -Dselenium.url and -Dapp.url if running using the TestNG plugin for Eclipse to pass in parameters.  We also configure WebDriver to automatically wait 10 seconds for events to occur.  Don't forget to tell people writing tests that they don't need to do this again in their code!

This configuration gives us a handful of options, developers can boot up a Selenium server by issuing "mvn -Dselenium.background=false integration-test", then run tests either in Eclipse or by calling surefire directly with "mvn surefire:integration-test".

Reporting the News

To recap, we can start a container, deploy our application, wait for it to start, then launch Selenium, run some tests and shut Selenium down when our tests have finished.  Our final goal is to report an announce the results of our test. Generating reports is pretty easy, we'll just add the following to our POM:

<reporting>
    <plugins>
        <plugin>
            <groupId>org.apache.maven.plugins</groupId>
            <artifactId>maven-surefire-report-plugin</artifactId>
            <version>2.5</version>
            <reportSets>
                <reportSet>
                    <id>integration-tests</id>
                    <reports>
                        <report>report-only</report>
                    </reports>
                    <configuration>
                        <outputName>failsafe-report</outputName>
                        <reportsDirectories>
                            <reportsDirectory>${project.build.directory}/failsafe-reports</reportsDirectory>
                        </reportsDirectories>
                    </configuration>
                </reportSet>
            </reportSets>
        </plugin>
    </plugins>
</reporting>
 

Wow, that was easy.  In Hudson, we configure the selenium-tests project to run the following Maven Goals: integration-test site-deploy failsafe:verify. This asks Maven to perform the tests, deploy the site and then fail if the tests failed.

I've also configured Hudson to send an email out if this build fails, that email contains the URL directly to the Selenium Failsafe report!

Now we have an automated deployment mechanism, Selenium tests and reporting.

Awesome!

Anyone who has had the pleasure of customizing the Commerce Manager knows that it is a complicated piece of software with a lot of moving parts. 

However, with the release of 6.2.1 combined with the new binary-based architecture and a number of clients going through an upgrade process, it has become even more complex with the introduction of patch fragments and a new approach to customizations.

For this blog post, I wanted to demonstrate a handy Eclipse plug-in that lets you visualize the dependencies between the various plug-ins within the Commerce Manager.  This is really useful for quickly discovering which plug-ins/bundles are dependent on each other.

The plug-in is called "Plug-In Dependency Graph Plugin". You can download it from http://testdrivenguy.blogspot.com/2009/05/eclipse-plug-in-dependency-graph.html. Then just unpack it into your Eclipse plugins folder and restart Eclipse.

Open the "Graph Plug-in Dependencies" view, click the search icon in the top right corner, and type in the name of the plugin you want to check out. In this example, I am interested in the our customized admin.configuration plugin.

You can see that there are a lot of dependencies between all the plugins, but by highlighting the admin.configuration plugin I can quickly see which plugins are directly referenced by it.

Alternatively, if you want to see who is directly referencing your plugin, you can show the callers:

In this case, not too interesting.  Let's take a look at com.elasticpath.cmclient.core:

A little more interesting. This plugin can help you quickly track down configuration issues in your plug-in manifest files when developing using the new patch fragment approach.

In future posts, I will provide more technical details as to our approach in implementing the patch fragment architecture including technical challenges encountered and our solutions to these issues.

Question: what errors are your customers seeing during checkout? Standard checkout optimization has us creating funnels in our analytics packages, identifying the choke points, and A/B testing variations of poorly performing pages. But how do we decide what changes to make to the page? Where are shoppers having problems? What if we've moved to a single page checkout – how do we identify problem areas?

Capturing the Information

One way to help answer these questions is to track the validation errors that customers see by using Google Analytics' event tracking utilities. Event tracking allows us to store arbitrary information in a hierarchical format within Google Analytics, so that we can examine our data in typical Google Analytics fashion. Using a relatively simple snippet of code in our Elastic Path velocity templates, we can store our form validation errors:

#set($errorsObj = $springMacroRequestContext.getErrors("expressCheckoutFormBean"))
#if ($errorsObj.errorCount > 0)

     <script type="text/javascript">
          #if($errorsObj.globalErrorCount > 0)
               #foreach($error in $errorsObj.globalErrors)
                    _gaq.push(["_trackEvent", "Checkout Error", "$error.code", "global"]);
               #end
          #end


          #if($errorsObj.fieldErrorCount > 0)
               #foreach($error in $errorsObj.fieldErrors)
                    _gaq.push(["_trackEvent", "Checkout Error", "$error.code", "$error.field"]);
               #end
          #end
     </script>
#end

You'll need to substitute the name of your own form bean for the call to getErrors(). If you have a multi-page checkout process, you may want to use a different event tracking label for each page, rather than just Checkout Error. Note also that this code uses the newer asynchronous format of the Google Analytics tracker – if you are using the old format, call the _trackEvent function of your GA tracker object.

Analyzing the Data

Now that we have the event tracking in place, what can we learn from the data? If we navigate in Google Analytics to Content → Event Tracking → Categories → Checkout Error, we can see exactly which errors customers experienced (one thing to keep in mind: every error will be counted, so one page view can result in multiple errors being recorded). More interesting though is the Ecommerce tab, which shows us in the Transactions column whether or not customers eventually checked out:

This is over a 10 day time period. Not surprisingly, missing required fields are the most common error encountered. If we drill down into errors.required we can see exactly which required fields are tripping visitors up:

The big winner is CVV code! Fortunately only 6 of 128 people failed to eventually checkout, and it's possible that not all of those 6 were legitimate shoppers. Still, we may be able to make the checkout process smoother if we explain what CVV is and where to find it. A surprising number of people seem to have left off their credit card number as well. Potentially customers are simply missing the credit card section altogether. Now we have some ideas for A/B tests we can run.

Other problem fields seem to be email and phone number. Not too shocking; people are wary about giving these out and were perhaps hoping they weren't actually required. Maybe we should do a better job of explaining why we need this information.

Looking back at the first Checkout Error screen, the big conversion drop-offs seem to be in places we'd expect: rejected credit card authorizations, incorrect address data, asking the customer to phone in their order, and no inventory. Hmm, what's this errors.whitespace?

The validation rule is "no leading or trailing whitespace allowed", so someone probably had a space after their email address, couldn't figure out the error, and gave up. Stripping the whitespace prior to validation makes more sense and might have saved the order. From a net revenue standpoint this improvement isn't high on the list, but fixing it is 10 minutes of our time, and we can check to see if this rule applies to any other fields.

There is a great deal more information to delve into here, and over time we will build up a history that will allow us to spot new problems as they crop up. By adding a short and simple snippet of code, we've gained greater insight into our customers' behavior, and hopefully discovered some ways to increase our conversion rate and create a smoother checkout process.

David Minor works for women's sportswear retailer Team Estrogen, an Elastic Path customer.

Apache HTTP Server is a very effective tool for caching static content and, if configured properly, can improve performance of your Elastic Path deployment by up to 30%! Furthermore, Apache does a great job of load balancing a cluster of storefront nodes, giving you even more throughput and scalability, without resorting to expensive hardware load balancers. Obviously, Apache will never perform like a hardware load balancer, but it is a little more affordable (read: free). So really, what more can you ask for from an HTTP server?

In this post, we'll look at using Apache to load balance our storefront servers. We'll also look at enabling caching of static content at the Apache level, removing a lot of network and CPU load from our application servers and giving a faster load time to browsers. Before we begin, make sure you have the following:

• Apache HTTP Server 2.2.10+ with either JBoss 4.2+ or Tomcat 5.5+ (using Apache with WebLogic is more complicated and requires the use of a specific Oracle-WebLogic Apache plug-in.)
• Apache has been built with the following modules: mod_proxy, mod_proxy_ajp, mod_proxy_balancer, mod_cache, mod_disk_cache.

Configuring a Proxy and Static Content Cache

Let's start by creating a proxy server and caching static content at the Apache level. This is relatively easy to set up, but important to understand before moving on to load balancing. We'll assume Apache is the front-most facing component to the user's browser. The architecture will look something like the following diagram.

Let's examine a request working it's way through this architecture. A typical first request from a shopper's browser, such as viewing a product page, will flow through Apache (bypassing all caches since they're empty) and arrive at the application server. The application server will gather and serve the necessary HTML and subsequent embedded objects (images, js, css, etc). These objects will pass back through Apache and to the user's browser. The key process here, however, is that as these static objects pass back through Apache, Apache will cache them based on their cache control headers.

When a request comes in for the same product page (or any request for the same set of static HTML objects), Apache will serve the static objects straight back to the user's browser from its cache. Only the dynamic HTML and other dynamic content will come from the app server. Although a second load of the same page on the same user's browser will already be cached at the user's browser level, it will be very useful for new sessions that have an empty browser cache.

Unfortunately, there are a couple issues we need to think about before we can implement this setup, such as:

• How do we communicate between Apache and the app server?
• What protocol do we use between Apache and the app server, HTTP or AJP?
• How do we support Acegi security, which is required by the storefront application servers?

Don't worry! We did a fair amount of performance testing to answer these questions, and came up with the following diagram.

The key here, is that a) we're using AJP between Apache and the app server, a fast binary protocol, and b) we're using two separate AJP connectors on the app server, one non-secure for HTTP traffic and one considered "secure" for HTTPS traffic. This allows Acegi to know that a request is "secure" so that it will not try to redirect endlessly to a secure port (a typical problem we see). I'm putting "secure" in quotes because it's really no different than the insecure channel (it's not encrypted). It simply has additional header information stating it's a secure channel.

In order to implement this, there are a number of items to configure such as Apache's mod_proxy and mod_cache, as well as any cache control configuration that needs to be done on the application server.

mod_proxy

We need to allow requests that come in to Apache to pass through to the application server and then return to the user. This is done using Apache's mod_proxy module. The full mod_proxy documentation is here: http://httpd.apache.org/docs/2.2/mod/mod_proxy.html. It's a recommended read. We'll also be using the mod_proxy_ajp module for AJP support.

The first step is to enable the two AJP connectors on the application server, in server.xml (or jboss-server.xml):

<Connector enableLookups="false" port="8009" protocol="AJP/1.3"/>
<Connector enableLookups="false" port="8010" protocol="AJP/1.3" scheme="https" secure="true"/>

Note the secure parameters for port 8010. This fools Acegi into thinking that anything coming over this port with AJP is a secure connection and it will not redirect it.

The second step is to ensure Acegi knows it may receive connections over port 80 and its secure mapped port is then 443 (the typical HTTP and HTTPS ports). To do this, we edit the storefront web app's WEB-INF/conf/spring/security/acegi.xml file and add an additional port mapping to the portMapper bean as follows:

    <!-- port # are specified in default.xml -->
    <bean id="portMapper" class="org.acegisecurity.util.PortMapperImpl">
        <property name="portMappings">
            <map>
                <entry key="80"><value>443</value></entry>
                <entry key="8080"><value>8443</value></entry>
            </map>
        </property>
    </bean>

In the third and final step, we want to configure the HTTP and HTTPS virtual hosts on Apache to listen to ports 80 and 443.

LoadModule proxy_ajp_module modules/mod_proxy_ajp.so

<VirtualHost 10.10.90.54:80>
        ServerName 10.10.90.54
        ProxyPreserveHost On
        ProxyPass /storefront ajp://10.10.90.54:8009/storefront keepalive=On
</VirtualHost>

<VirtualHost 10.10.90.54:443>
        ServerName 10.10.90.54
        # Enable/Disable SSL for this virtual host if you want to terminate SSL here
        ProxyPreserveHost On
        ProxyPass /storefront ajp://10.10.90.54:8010/storefront keepalive=On
</VirtualHost>

There's a lot going on here, so let's have a look at the HTTPS:443 virtual host as it's the more complex one here:

1. Clearly, one would want to configure the virtual hosts to listen on the specific machine's port.
2. Within here is where we would do any SSL termination before passing the request over AJP to the app server.
3. "ProxyPreserveHost On" ensures the Host header is maintained as it's passed to the app server. This is required for Elastic Path 6.1 and later to be able to handle multi-store requests.
4. The ProxyPass directive is the key here. This tells Apache to pass any requests coming in matching /storefront to the app server's AJP connector under /storefront.
5. There are a large number of options for this directive, including maintaining keepalive, as we've done here.
6. Note that the storefront server doesn't have to be the localhost. We'll see this later when we being load balancing.

At this point, after rebooting, you should be able to hit Apache on port 80 and pull up your storefront.

mod_cache

Next, we want to cache any static content we can on the Apache side. To do this, we'll use mod_cache, or more specifically mod_disk_cache. There is also mod_mem_cache, which is a memory based cache, but we've actually found better performance results with mod_disk_cache, plus the persistence of all cache files is a plus.

Adding the httpd.conf directives for a disk cache is fairly straightforward. Let's try the following:

CacheEnable disk /storefront/
CacheRoot /var/www/cache
CacheDirLevels 5
CacheDirLength 2
CacheIgnoreHeaders Set-Cookie

Looking at the lines in detail:

1. Enable the disk cache on the URL /storefront/.
2. Specify the cache location on the local disk, in this case /var/www/cache. You'll want to make sure the Apache user can write to that directory.
3. The number of directory levels in the cache tree structure.
4. The number of characters for each directory.
5. Finally, we specify which headers we DO NOT want to cache. This is essential. If we don't set this for cookies, we will end up getting someone else's session!

At this point, after rebooting Apache, we will begin to cache any static objects with cache control headers. In order to expand on what is (or isn't cached), let's move on to the next section.

Cache Control Header Config

Finally, we want the application server, or more specifically, the deployed applications, to tell Apache if there's anything to cache. This is typically done by using cache control headers, such as max-age.

For the storefront web application, you can use the Caching Control Filter to add the max-age cache control header to requests for specific types of content (based on URL patterns). The Caching Control Filter configuration is in the storefront's conf/spring/web/filter-config.xml file, in the cachingControlFilter bean definition. The cachingControlEntries list contains bean definitions that represent the URL patterns to test and max-age value to set.

The following is an example of caching all /renderImage.image dynamic image calls, all /template-resources/ calls (css, js, etc) and any dynamic content assets under /content/:

<bean id="cachingControlFilter"

     class="com.elasticpath.commons.filter.impl.CachingControlFilter">
     <property name="cachingControlEntries">
          <list>
               <bean class="com.elasticpath.commons.filter.impl.CachingControlFilter$CachingControlEntry">
                    <property name="urlPattern">
                         <value>^.*renderImage\.image.*$</value>
                    </property>
                    <property name="maxAge">
                         <value>86400</value>
                    </property>
               </bean>
               <bean class="com.elasticpath.commons.filter.impl.CachingControlFilter$CachingControlEntry">
                    <property name="urlPattern">
                         <value>^.*template-resources.*$</value>
                    </property>
                    <property name="maxAge">
                         <value>86400</value>
                    </property>
               </bean>
               <bean class="com.elasticpath.commons.filter.impl.CachingControlFilter$CachingControlEntry">
                          <property name="urlPattern">
                                  <value>^.*content.*$</value>
                    </property>
                    <property name="maxAge">
                         <value>86400</value>
                    </property>
               </bean>
          </list>
     </property>
</bean>

Now, after restarting Apache, you should have a fully functioning Apache proxy with proper caching of static content.

Configuring Load Balancing

Load balancing is an easy extension once our proxy is set up. Essentially, with load balancing, instead of passing the request through to the same machine each time, we pass it to a cluster of machines (two or more) based on a certain algorithm. I recommend reading the complete Apache documentation on mod_proxy_balancer, which is the module we'll use to enable load balancing. It can be found here: http://httpd.apache.org/docs/2.2/mod/mod_proxy_balancer.html

Let's first lay out the Apache configuration, adding to our existing VirtualHost entries.

<VirtualHost 10.10.90.54:80>

        ServerName 10.10.90.54

        # ProxyPreserveHost On
        RequestHeader set Host mars.elasticpath.net

        <Proxy balancer://tomcatservers>
                BalancerMember ajp://localhost:9009 route=node1 loadfactor=90
                BalancerMember ajp://10.10.90.51:9009 route=node2 loadfactor=100
                BalancerMember ajp://10.10.90.52:9009 route=node3 loadfactor=100
                BalancerMember ajp://10.10.90.53:9009 route=node4 loadfactor=100
        </Proxy>

        ProxyPass /storefront balancer://tomcatservers/storefront stickysession=JSESSIONID nofailover=Off
        ProxyPass /server-status !

</VirtualHost>

<VirtualHost 10.10.90.54:443>

        ServerName 10.10.90.54

        LogLevel warn
        #CustomLog logs/ssl_request_log "%t %h %{SSL_PROTOCOL}x %{SSL_CIPHER}x \"%r\" %b"
        LogFormat "%h %l %u %t \"%r\" %>s %b" common
        CustomLog logs/ssl_access_log common
        ErrorLog logs/ssl_error_log

        #   SSL Engine Switch:
        #   Enable/Disable SSL for this virtual host.
        SSLEngine on
        SSLCipherSuite ALL:!ADH:!EXPORT56:RC4+RSA:+HIGH:+MEDIUM:+LOW:+SSLv2:+EXP:+eNULL
        SSLCertificateFile "/usr/local/apache2/conf/server.crt"
        SSLCertificateKeyFile "/usr/local/apache2/conf/server.key"

        #DocumentRoot    "/var/www/html/one"

        # ProxyPreserveHost On 
        RequestHeader set HOST mars.elasticpath.net

        SetEnvIf User-Agent ".*MSIE.*" nokeepalive ssl-unclean-shutdown downgrade-1.0 force-response-1.0

        <Proxy balancer://tomcatservers-ssl>
                BalancerMember ajp://localhost:9010 route=node1 loadfactor=90
                BalancerMember ajp://10.10.90.51:9010 route=node2 loadfactor=100
                BalancerMember ajp://10.10.90.52:9010 route=node3 loadfactor=100
                BalancerMember ajp://10.10.90.53:9010 route=node4 loadfactor=100
        </Proxy>

        ProxyPass /storefront balancer://tomcatservers-ssl/storefront stickysession=JSESSIONID nofailover=off

</VirtualHost>

We've seen the VirtualHost entries before, so let's just look at the Proxy balancer configuration in detail. We'll zoom in on this below for the insecure, port 80 connector:

...
        <Proxy balancer://tomcatservers>
                BalancerMember ajp://localhost:9009 route=node1 loadfactor=80
                BalancerMember ajp://10.10.90.51:9009 route=node2 loadfactor=100
                BalancerMember ajp://10.10.90.52:9009 route=node3 loadfactor=100
                BalancerMember ajp://10.10.90.53:9009 route=node4 loadfactor=100
        </Proxy>

        ProxyPass /storefront balancer://tomcatservers/storefront stickysession=JSESSIONID nofailover=Off
        ProxyPass /server-status !
...

Within the Proxy balancer directive, we've named our balancer <Proxy balancer://tomcatservers> and defined our load balancer members, for example BalancerMember ajp://10.10.90.51:9009 route=node2 loadfactor=100. In this case, we have 4 storefronts included, 1 being on the localhost (the same machine as the Apache install). We've opened up AJP port 9009 on these machines and configured all to have an even load factor, except the first node, which has slightly lower factor to give breathing room for Apache on the same machine.

The next directive, ProxyPass /storefront balancer://tomcatservers/storefront, we specify our ProxyPass to allow requests to /storefront* to pass to the balancer's /storefront*.  Note that we're also specifying the cookie name we want to keep stuck to each storefront node, so subsequent requests return to the same node. In our case, this is the JSESSIONID.

The last key setting in here is the route=nodeN on each BalancerMember. This is the name you configure for a node's jvmRoute within the app's server.xml. This allows Apache and the application server to identify which requests will go to which node. Without this setting (and/or the stickysession setting), the user's session may bounce between storefront nodes. This will cause strange behavior, like gettin bounced back to the homepage.

To set the jvmRoute within the server.xml, look for a commented-out line like the following:

<!-- You should set jvmRoute to support load-balancing via AJP ie :
 <Engine name="Catalina" defaultHost="localhost" jvmRoute="node1">         
-->   

Uncomment this and change jvmRoute="" to be the same as your BalancerMember entry (or vice versa). The same configuration as above is done for the secure connectors, which, in this case are on port 9010.

After rebooting Apache, you should be getting load balanced to a specific node in the cluster and stay on that node for subsequent requests. Your HTML assets will also be getting cached at the Apache layer as they pass through the proxy.

Now you can cache and load balance storefront servers with Apache HTTP Server. Go ahead and try it. Once you're set up, I would recommend tailing your Apache and app server access logs to watch your requests pass through Apache and your app server and ensure they're using sticky sessions correctly. Increasing the access log level on Apache and the app server to output cookie names/values is handy if you need to debug any sticky session config issues.

Some Final Considerations

• There are some known issues around keep-alive and some older versions of Apache HTTP and Tomcat where the AJP connections between the two will not get released, causing the connection pool to fill and not allow new requests.
• Consider using Apache's htcacheclean, which runs as a daemon or a one-time job, to control the size of your Apache cache on the disk. If your website has a small, finite number of cacheable HTML objects, this typically isn't a huge issue. On the other hand, if you have many GBs of assets and want to keep your cache to, say, 500 MB, htcacheclean is your tool. See the documentation for full details: http://httpd.apache.org/docs/2.2/programs/htcacheclean.html
• Test, test, test. Make sure you do proper functional testing on a staging environment to ensure there are no strange redirects or odd behavior after putting another layer between your ecommerce site and the user. And just as importantly, proper performance testing will ensure there are no capacity issues between Apache and the app server. This will allow you to fine tune your connection pools for maximum performance, both on the Apache side and the app server side.

A while back, I wrote about our decision to change to a two-page checkout process, with the main goal being to reduce checkout process abandonment. We piloted this checkout process on the Hockey Canada store and the results were extremely positive, but we weren't content to sit on our laurels. So, when we started re-designing the official Vancouver 2010 Olympic store, we challenged ourselves to take it to the next level -- and we cut the checkout process down to just one page.

Structurally, the new single-page checkout looks very much like the two-page checkout, with shipping information first, followed by billing and confirmation. The Elastic Path Commerce platform is flexible enough to handle multiple checkout process flows for the same store, so there was no significant Google Website Optimizer integration required to make this work.

Variant A (Control): Multi-page Checkout

Page 1 (sign in):

Page 2 (shipping address):

Page 3 (shipping method):

Page 4 (billing & review):

Page 5 (receipt)

Variant B: Single Page Checkout

Page 1 (shipping, billing):

Page 2 (receipt and optional user registration form):

In A/B split testing, 50% of site traffic was redirected to the OOTB checkout, while the other 50% was served the new single-page checkout. By the time we reached 300 transactions, the winner was clear, and we stopped the experiment after 606 transactions. Google Website Optimizer concluded that the single-page checkout outperformed the out-of-the-box checkout by a whopping 21.8%. But what does that 21.8% really mean? GWO only counts goal conversions and does not link to any ecommerce data on Google Analytics, so we used Advanced Segments to get this data passed on to Google Analytics.

We defined two Advanced Segments by creating the following expressions:

• Multi-step checkout: /(?:checkout|shipping-address|billing-address|delivery-options|billing-and-review)\.html.*
• Single page checkout: /check-out\.html.*

This allowed us to track metrics like Average Order Value and Conversion Rate for each experiment variation.

Here's what we observed:

• Successful completion rate for the entire checkout process increased by 257.26%.
• Overall site conversion rate increased by 0.54%.
• We also observed some unexpected improvements during this experiment, like an increase of 8.54% in the average order value!

While these numbers are impressive, they should not be used as the sole indicator of how single-page checkout performs. This is just what we observed when changing from the standard four-page checkout to a single-page checkout process on the Vancouver 2010 Olympic Store. Your mileage may vary, depending on your product, target market, etc. There's no silver bullet checkout process that works best for all business models. Doing your own A/B split testing will give you a better idea of what kinds of numbers you can expect.