Overview:

Elastic Path 6.2.2 comes with a little known project called "com.elasticpath.test.application". This project is used as a connector to enable integration testing by exposing the spring application context to hook up Elastic Path core services for retrieving, updating, and removing EP domain objects. This article will go through the steps on how to use this project for integration testing.

Why?

Our client is using EP's web services to perform shopping related operations, and while we can test the web service calls with a framework like Groovy, what we also needed was the ability to check the backend to ensure the result of the web service call has been persisted in the database. We also needed to manipulate the data objects so we could test the edge cases through the web service. We could have written raw SQL statements to insert these rows, but then again you would have to insert the object graph of these domain objects. The downside to this is if one were to add a new column or remove an existing column, these SQL statements will have to be changed.

The Solution:

We rely on one of the projects that come packaged seperately in 6.2.2: com.elasticpath.test.application. This project will give us hooks into Spring and the service layer which we can then use to manipulate the domain objects. In addition, it provides us with a few Factory methods which we can use to create and persist certain domain objects for testing.This project was originally designed for use with FIT tests, but with a little bit of tweaking we can use it to our advantage.

The Setup:

- Elastic Path 6.2.2, taking advantage of the binary-based development approach

- Java 6

- Maven 2.2.1

The Details:

Step 1: Create your Integration test project.

In the pom file reference these jars. Here I am assuming you have a core extension project already, such that it is available for reference in this test project.

     <dependencies>
        ...
        <dependency>
            <groupId>com.elasticpath</groupId>
            <artifactId>com.elasticpath.test.application</artifactId>
            <version>6.2.2</version>
            <scope>compile</scope>
        </dependency>
        <dependency>
            <groupId><your group id></groupId>
            <artifactId><your core artifact id></artifactId>
            <version><your version #></version>
            <scope>compile</scope>
        </dependency>
        <dependency>
            <groupId>com.elasticpath</groupId>
            <artifactId>com.elasticpath.core.resources</artifactId>
            <version>6.2.2</version>
            <scope>compile</scope>
        </dependency>
          ...
      </dependencies>

NOTE: We need the com.elasticpath.core.resources package because it contains the out-of-the-box Spring bean definitions that we will use later.

Step 2: Copy the files from the com.elasticpath.core.itest project into our new project.

The reason we need this is because we need to set up the database configuration and spring definitions specific for integration tests. Copy the two files integrationtests/test_application.config and integration-core-context.xml into your Maven test project's src/main/resources folder.

Step 3: Modify your test_application.config file.

Change the setting in the file to correspond to the line below. We need to perform this change because all of our required spring definitions come from this file.

context.definitions_file=integration-core-context.xml

Step 4: Modify your integration-core-context.xml file.

Here we will need to do a couple of things. First, append this line to the end of the file:

    <import resource="classpath*:META-INF/conf/plugin.xml" />

This will import your definition from your core extension project.

The next step involves an examination of the integration-core-context.xml file. Due to the fact that there are still some FIT test references in this file, we will have to look for them and remove them as neccessary. Remove the beans with the following IDs:

importJobRunnerBaseAmount
importJobRunnerBaseAmountForFit
baseAmountDtoInsertUpdateImporterForFit
contentWrapperLoader
contentWrapperRepository
velocityRenderer
checkoutEventHandler

Step 5: Modify the com.elasticpath.test.application project.

A couple of modifications to the com.elasticpath.test.application project are required in order for us to use it. First off, we will need to add the following to TestApplicationContext.java:

A.) Change the variable TEST_APPLICATION_CONFIG_FILE to read as shown on the next line. This is because the location of the file has changed.

    private static final String TEST_APPLICATION_CONFIG_FILE = "test_application.config";

B.) Change the method overrideAllDefinitionsToLazy() to correspond to the code block below. We need to do this because we want to load the additional prototype beans defined by the BeanRegistrar.

    private void overrideAllDefinitionsToLazy() {
        for (int i = 0; i < beanFactory.getBeanDefinitionNames().length; i++) {
            String name = beanFactory.getBeanDefinitionNames()[i];
            if (name != null && !name.startsWith("beanRegistrar")) {
                AbstractBeanDefinition beanDefinition = (AbstractBeanDefinition) beanFactory.getBeanDefinition(name);
                beanDefinition.setLazyInit(true);
            }
        }
    }

C.) Inside the initializeBeanFactory() method, make the change described below. We need to do this because we are now loading the file from the Classpath.

        beanFactory = new XmlBeanFactory(new FileSystemResource(definitionsFile));  

          to

        beanFactory = new XmlBeanFactory(new ClassPathResource(definitionsFile));

D.) Replace the method getApplicationProperties() with the following code block below. We need to do this because we are now loading the file from the Classpath.

    private Properties getApplicationProperties() {
        Properties properties = new Properties();
        try {
            InputStream inStream = this.getClass().getClassLoader().getResourceAsStream(TEST_APPLICATION_CONFIG_FILE);
            properties.load(inStream);
            
        } catch (IOException e) {
            throw new TestApplicationException("Failed to load properties from file:" + TEST_APPLICATION_CONFIG_FILE, e);
        }
        return properties;
    }

Step 6: Rebuild the com.elasticpath.test.application project by navigating to the project root directory and issuing the following command:

ant clean jar

Step 7: Now test that your integration project works by creating a test called MyTest.java under src/test/java in the new integration test project and copy the following into that file:

public class CreateWarehouseTest {
 
  private StoreTestPersister storeTestPersister;
 
  @Before
  public void setUp() {
    TestApplicationContext tac = TestApplicationContext.getInstance();
    tac.useDb();
 
    storeTestPersister = tac.getPersistersFactory()
        .getStoreTestPersister();
  }
 
  @Test
  public void testCreateWarehouse() {
    storeTestPersister.persistDefaultWarehouse();
  }

Step 8: Now configure your test_application.config file in your integration test project to point to your database and the initial database script location, and then run the test you just created.

db.script.dir=<your db script dir>

You should see the warehouse being persisted to the database.

(Or: How to Copy and Paste XML Your Way to Greatness)

We have a truckload of Selenium tests that poke and probe away at the user interface of our project here at EP. We realized it would be awesome if this testing could just automagically happen and let us know the results. So we took a look at our Maven build and realized the steps we wanted to automate were:

1. Check out our source code, build the latest set of artifacts and deploy them to our internal Maven repository.
2. Stop the web application running on our QA server environment.
3. Recreate our test MySQL database with our default schema, apply any change sets needed to bring it to the latest version and load our QA data.
4. Update the test data to reflect our QA server environment.
5. Deploy our demo assets to a known location.
6. Start our web application and wait for it to fully start.
7. Launch Selenium and run our tests against our application on Firefox on our Linux Hudson slave.
8. Record and announce the results of the tests.

In this blog post, I'm going to cover how we automated the last three steps.

The first thing we needed to do was find a tool that could automate configuring a web app container and deploying our WAR files. This tool would need to support multiple web containers without having to write custom scripts for each one. After doing some research, we decided to go with the fairly capable Cargo project. It has a fairly mature Maven plugin and good support for our preferred container, Tomcat, and it's come a long way since its inception. On our side, all we'd need to do is write a simple Groovy script to wait until our web application is in a ready-to-test state.

Our particular application really benefits from the new WebDriver features in Selenium 2.0, and while we'd love to take advantage of the Selenium Plugin for Maven, it's tied to Selenium 1.0, so we can't fully use it yet. We can however use it to bootstrap an X server on our Hudson build slave.

Our Selenium tests are written using TestNG and stored in their own Maven artifact. We can easily bind the Maven Failsafe Plugin to run during our integration-test phase, but we'd like to publish the output to our internal reporting site, so we can take a look at the last results. Luckily, we can use the Maven Surefire Report plugin to generate our site. Lastly, we'll get Hudson to announce the results of our test. (The page on the Maven build lifecycle is invaluable when figuring out when to run different plugins.)

So, let's get started!

Properties

To make Cargo and Selenium more useful, we pushed a handful of settings into a parent POM. This lets us override values and use the same artifacts for our own local development testing.

<properties>
    <cargo.jvmargs>-XX:MaxPermSize=512M -Xmx1024m</cargo.jvmargs>
    <cargo.servlet.port>8080</cargo.servlet.port>
    <cargo.tomcat.ajp.port>8009</cargo.tomcat.ajp.port>
    <cargo.rmi.port>1099</cargo.rmi.port>
    <cargo.tomcat.shutdown.port>8205</cargo.tomcat.shutdown.port>
    <cargo.wait>true</cargo.wait>

    <demo.hostname>10.9.8.7</demo.hostname>

    <searchserver.context>searchserver</searchserver.context>
    <ourapp.context>ourapp</ourapp.context>
    <cmserver.context>cmserver</cmserver.context>
    <storefront.context>storefront</storefront.context>

    <!-- Absolute -->
    <assets.directory>${user.home}/ep-assets</assets.directory>

    <jdbc.driver>com.mysql.jdbc.Driver</jdbc.driver>
    <jdbc.host>10.9.8.7:3306</jdbc.host>
    <jdbc.db>OURAPP_DB</jdbc.db>
    <jdbc.username>atwill</jdbc.username>
    <jdbc.password>grep</jdbc.password>
</properties>

Okay, that's more than a handful. but let's go over them really quickly. The cargo.* properties are there to let us easily have multiple Tomcats running on the same IP, the *.context properties tell Cargo where to deploy our various WAR files. The cargo.wait property tells Cargo to pause after it starts the application server. If set to false, Cargo will proceed along the Maven build lifecycle.  The assets.directory is used by steps 4 and 5.  The demo.hostname is the public hostname (or IP in our case) for our deployment (you'll see why we do this as we move along). The jdbc.* properties describe our JDBC DataSource. We've broken the information up so that we can use it in steps 3, 4 and 6. As you can imagine, it's pretty easy to override these properties in your local settings.xml and use Cargo to start a local app server for you - lucky you!

Containing the Excitement

Our first goal is to start a container, deploy our artifacts and wait for them to all start. For our situation, we felt it was best to have a separate container Maven artifact (with pom packaging) dedicated to steps 5 and 6. So, first we'll declare the WAR and JAR files that we depend on for the container. We included the JDBC driver and the JAI JARs for our Storefront too. We've already declared the versions for all these artifacts in a parent POM.

....            
           <dependencies> 
                <!-- WARs to be deployed by Cargo (also add them in cargo below) -->
                <dependency>
                        <groupId>com.elasticpath.mdm</groupId>
                        <artifactId>ourapp-war</artifactId>
                        <type>war</type>
                </dependency>

                <dependency>
                        <groupId>com.elasticpath.mdm</groupId>
                        <artifactId>searchserver</artifactId>
                        <type>war</type>
                </dependency>

                <dependency>
                        <groupId>com.elasticpath.mdm</groupId>
                        <artifactId>cmserver</artifactId>
                        <type>war</type>
                </dependency>

                <dependency>
                        <groupId>com.elasticpath.mdm</groupId>
                        <artifactId>storefront</artifactId>
                        <type>war</type>
                </dependency>

                <!-- Dependencies provided to Cargo -->
                <dependency>
                        <groupId>mysql</groupId>
                        <artifactId>mysql-connector-java</artifactId>
                        <scope>provided</scope>
                </dependency>

                <dependency>
                        <groupId>javax.media</groupId>
                        <artifactId>jai-core</artifactId>
                        <scope>provided</scope>
                </dependency>

                <dependency>
                        <groupId>com.sun.media</groupId>
                        <artifactId>jai-codec</artifactId>
                        <scope>provided</scope>
                </dependency>
        </dependencies>

Next up, we'll start Cargo in the pre-integration-test phase and stop it when we hit the post-integration-test phase. If you're copying and pasting this XML, you'll want to paste the next three sections in sequence into your POM, but we'll cover them individually.

<build>
    <plugins>
        <plugin>
            <groupId>org.codehaus.cargo</groupId>
            <artifactId>cargo-maven2-plugin</artifactId>
            <version>1.0.1-beta-2</version>
            <executions>
                    <execution>
                        <id>start-container</id>
                        <phase>pre-integration-test</phase>
                        <goals>
                            <goal>start</goal>
                        </goals>
                    </execution>
                    <execution>
                        <id>stop-container</id>
                        <phase>post-integration-test</phase>
                        <goals>
                            <goal>stop</goal>
                        </goals>
                    </execution>
            </executions>

            <configuration>
                <container>
                    <dependencies>
                        <dependency>
                            <groupId>mysql</groupId>
                            <artifactId>mysql-connector-java</artifactId>
                        </dependency>

                        <dependency>
                            <groupId>javax.media</groupId>
                            <artifactId>jai-core</artifactId>
                        </dependency>

                        <dependency>
                            <groupId>com.sun.media</groupId>
                            <artifactId>jai-codec</artifactId>
                        </dependency>
                    </dependencies>

                    <containerId>tomcat5x</containerId>
                    <zipUrlInstaller>
                        <url>file:${project.basedir}/apache-tomcat-5.5.29.zip</url>
                        <installDir>${installDir}</installDir>
                    </zipUrlInstaller>
                </container>

In the previous example, we bind the plugin to the lifecycle and tell Cargo to include the JDBC driver and JAI JARs into the classpath. (We won't get native acceleration for JAI, but that's fine for our purposes.) You'll probably notice the tricky thing we did for the zipUrlInstaller. To remove a dependency from our startup, we've checked in a copy of Tomcat into our project. Let's take a look at the Cargo configuration:

<configuration>
    <home>${project.build.directory}/tomcat5x/container</home>
    <properties>
        <cargo.jvmargs>${cargo.jvmargs}</cargo.jvmargs>
        <cargo.logging>high</cargo.logging>
        <cargo.servlet.port>${cargo.servlet.port}</cargo.servlet.port>
        <cargo.tomcat.ajp.port>${cargo.tomcat.ajp.port}</cargo.tomcat.ajp.port>
        <cargo.rmi.port>${cargo.rmi.port}</cargo.rmi.port>
        <cargo.tomcat.shutdown.port>${cargo.tomcat.shutdown.port}</cargo.tomcat.shutdown.port>
        <!--
            Ampersands must be escaped such that they show up as "&" in
            the resulting XML file, so we use "&amp;" here.
        -->
        <cargo.datasource.datasource>
            cargo.datasource.url=jdbc:mysql://${jdbc.host}/${jdbc.db}?AutoReconnect=true&amp;amp;useUnicode=true&amp;amp;characterEncoding=utf-8|
            cargo.datasource.driver=${jdbc.driver}|
            cargo.datasource.username=${jdbc.username}|
            cargo.datasource.password=${jdbc.password}|
            cargo.datasource.type=javax.sql.DataSource|
            cargo.datasource.jndi=jdbc/epjndi
        </cargo.datasource.datasource>
    </properties>

Here we're passing in the cargo.* properties from our parent POM and we're asking Cargo to create a DataSource for us with the jdbc.* properties and add it to the JNDI tree.  For our situation, we're always using MySQL, so we can make a handful of assumptions in the URL.  Now that we've picked a container and configured it, we'll tell Cargo which artifacts we'd like to have deployed:

            <deployables>
                <!-- Applications to Deploy -->
                <deployable>
                    <groupId>com.elasticpath.mdm</groupId>
                    <artifactId>ourapp-war</artifactId>
                    <type>war</type>
                    <properties>
                            <context>/${ourapp.context}</context>
                    </properties>
                    <pingURL>http://${demo.hostname}:${cargo.servlet.port}/${ourapp.context}
                    </pingURL>
                    <pingTimeout>60000</pingTimeout>
                </deployable>
                <deployable>
                    <groupId>com.elasticpath.mdm</groupId>
                    <artifactId>searchserver</artifactId>
                    <type>war</type>
                    <properties>
                            <context>/${searchserver.context}</context>
                    </properties>
                    <pingURL>http://${demo.hostname}:${cargo.servlet.port}/${searchserver.context}/product/select?q=*:*
                    </pingURL>
                    <pingTimeout>60000</pingTimeout>
                </deployable>
                <deployable>
                    <groupId>com.elasticpath.mdm</groupId>
                    <artifactId>cmserver</artifactId>
                    <type>war</type>
                    <properties>
                            <context>/${cmserver.context}</context>
                    </properties>
                    <pingURL>http://${demo.hostname}:${cargo.servlet.port}/${cmserver.context}/
                    </pingURL>
                    <pingTimeout>60000</pingTimeout>
                </deployable>
                <deployable>
                    <groupId>com.elasticpath.mdm</groupId>
                    <artifactId>storefront</artifactId>
                    <type>war</type>
                    <properties>
                            <context>/${storefront.context}</context>
                    </properties>
                    <pingURL>http://${demo.hostname}:${cargo.servlet.port}/${storefront.context}/
                    </pingURL>
                    <pingTimeout>60000</pingTimeout>
                </deployable>
            </deployables>
        </configuration>
    </configuration>
</plugin>

This is pretty much textbook Cargo; you'll see our use of demo.hostname above in PingURL. Cargo will fetch the artifacts and deploy them into Tomcat waiting for the specified URL to not return an error, waiting up to pingTimeout milliseconds. Since we're just a demo application, the PingURL for the searchserver does a query for products. The problem here is that, while the searchserver may be available, the search indexes might not be built yet. So, we'll need something to check that the searchserver is not only started, but that its indexes are populated.

For this, we put together a simple Groovy script that waits until the searchserver returns products when queried. We put this in src/main/script/waitforstartup.groovy and it looks a little something like this:

import java.net.URL
import groovy.xml.StreamingMarkupBuilder


 def url = project.properties['searchserverUrl'];
 def timeout = project.properties['timeout'].toLong();

 /* For standalone testing, use something like:
  * 
  *     def url = "http://localhost:8080/searchserver/product/select?q=*:*"
  *     def timeout = 120*1000;
  */

def ready = false;
def text = "";
def first = true;

def begin = System.currentTimeMillis()

while (!ready) {
        
        if (System.currentTimeMillis() > (begin+timeout)) {
                // provided by gmaven
                fail("Timeout of "+timeout+"ms reached waiting for "+url+" to return at least one search result.")
        }
        
        if (!first) {
                Thread.sleep(10*1000);
        } else {
                first = false;
        }
        
        URL searchServer;
        try     {
                searchServer = new URL(url);
                text = searchServer.getText();
        } catch (e) {
                print "Server not yet ready, sleeping for 10 seconds. ("+e+")\n"
                continue;
        }
        
        if (text.size() == 0) {
                print "Not yet ready, sleeping for 10 seconds. (Empty HTTP response received)\n"
                continue;
        }
        
        try {
                def root = new XmlSlurper().parseText(text)
                
                if (numFound(root)!=0) {
                        ready = true;
                } else {
                        print "No elements returned in search result, sleeping for 10 seconds.\n"
                        continue;
                }
        } catch (e) {
                print "Trouble parsing XML result, sleeping for 10 seconds. ("+e+")\n"
                continue;
        }
}

def numFound(root) {
        return root.result.@numFound;
}

XmlSlurper was pretty neat (as used in numFound). Running it is pretty trivial with the GMaven plugin. You'll notice how we can easily pass in properties to the Groovy script via project.properties.

<plugin> 
    <groupId>org.codehaus.groovy.maven</groupId>
    <artifactId>gmaven-plugin</artifactId>
    <configuration>
        <properties>
            <timeout>300000</timeout>
            <searchserverUrl>http://${demo.hostname}:${cargo.servlet.port}/${searchserver.context}/product/select?q=*:*</searchserverUrl>
        </properties>
        <source>${basedir}/src/main/script/waitforstartup.groovy</source>
    </configuration>
</plugin>

The source element is relative to the current working directory, not your project's base directory, so it's a good idea to put in ${basedir} to prevent surprises later on.

Running mvn integration-test should scroll for a while as Maven downloads all required dependencies and eventually the container will start. If we run gmaven:execute, our Groovy script will run and happily poll our searchserver until it's ready!

This alone is a great set up for running builds of our project both on my development box and for our QA server!

In Hudson, I have one Job which spawns both targets in parallel.  The integration-test job never returns (until it's killed in step 2), but once the gmaven:execute target returns, it runs the Selenium tests.  Let's talk about those now...

Prepare for Take Off

Our second goal is to launch Selenium, run our tests and shut Selenium down.  It made sense for us to have a separate selenium-tests artifact for that purpose, let's start by taking a look at the properties we decided to set:

<properties>
    <selenium.version>2.0a4</selenium.version>
    <selenium.host>127.0.0.1</selenium.host>
    <selenium.port>14444</selenium.port>
    <selenium.DISPLAY>:17</selenium.DISPLAY>
    <selenium.background>true</selenium.background>
    <xvfb.option>-ac</xvfb.option>
    <seleniumUrl>http://${selenium.host}:${selenium.port}/wd/hub</seleniumUrl>
    <appUrl>http://${demo.hostname}:${cargo.servlet.port}/${ourapp.context}</appUrl>
</properties>

We're making use of the Maven Selenium Plugin to start up Xvfb on display :17, and we disable access control to Xvfb to work around any xauth issues we may run into.  Here you see another use of demo.hostname, passing it to Selenium as the application under test.  The selenium.background property allows us to run Selenium on the command line and keep the server running with -Dselenium.background=false, this running these tests during development makes testing a lot quicker!

Next we'll specify the dependencies we need to make this all happen:

<dependencies>
    <dependency>
        <groupId>org.seleniumhq.selenium</groupId>
        <artifactId>selenium</artifactId>
        <version>${selenium.version}</version>
    </dependency>

    <dependency>
        <groupId>org.testng</groupId>
        <artifactId>testng</artifactId>
        <version>5.11</version>
        <scope>test</scope>
        <classifier>jdk15</classifier>
    </dependency>

    <dependency>
        <groupId>org.seleniumhq.selenium</groupId>
        <artifactId>selenium-server</artifactId>
        <version>${selenium.version}</version>
    </dependency>
</dependencies>   
<repositories>
    <repository>
        <id>selenium-repository</id>
        <url>http://selenium.googlecode.com/svn/repository/</url>
    </repository>
</repositories>

No real surprises here, although I'm adding an extra repository just to pull down Selenium 2.0a4. You could easily mirror it locally in your own company-wide repository.

Now we need to start and stop Selenium.  Normally we'd use the Maven Selenium Plugin, but it's pinned to the older generation of Selenium. Instead of trying to be overly clever, we just ask Ant to start and stop Selenium during the pre-integration-test and post-integration-test phases, we pass in a DISPLAY variable which is ignored on non-Unix operating systems.

<build>
    <plugins>
        <plugin>
            <!--
                Use Ant to start and stop the selenium server - once the selenium
                plugin handles 2.0 we can get rid of this.
            -->
            <groupId>org.apache.maven.plugins</groupId>
            <artifactId>maven-antrun-plugin</artifactId>
            <executions>
                <execution>
                    <id>start-selenium</id>
                    <phase>pre-integration-test</phase>
                    <configuration>
                        <tasks>
                            <echo taskname="start-selenium"
                                message="Starting Selenium Server v${selenium.version} on ${selenium.port} offering display ${selenium.DISPLAY}" />
                            <java taskname="start-selenium"
                                jar="lib/selenium-server-standalone-${selenium.version}.jar"
                                fork="true" spawn="${selenium.background}">
                                <env key="DISPLAY" value="${selenium.DISPLAY}" />
                                <arg line="-timeout 30 -debug -browserSideLog -port ${selenium.port}" />
                            </java>
                        </tasks>
                    </configuration>
                    <goals>
                        <goal>run</goal>
                    </goals>
                </execution>
                <execution>
                    <id>stop-selenium</id>
                    <phase>post-integration-test</phase>
                    <configuration>
                        <tasks>
                            <echo message="Stopping Selenium Server" />
                            <get taskname="stop-selenium"
                                src="http://${selenium.host}:${selenium.port}/selenium-server/driver/?cmd=shutDown"
                                dest="${project.build.directory}/selenium-shutdown.txt"
                                ignoreerrors="true" />
                        </tasks>
                    </configuration>
                    <goals>
                        <goal>run</goal>
                    </goals>
                </execution>
            </executions>
        </plugin>

Okay, so we've configured Selenium to start and stop, now lets ask it to run our tests. We created an extremely simple TestNG suite in src/test/it/testng.xml:

<suite name="integration-tests">
     <test name="Everybody Everybody">
          <packages>
               <package name="com.elasticpath.mdm.tests.it" />
          </packages>
     </test>
</suite>

By convention, the Maven Failsafe Plugin will run src/test/java/<above packages>/*IT.java tests when called with integration-test and TestNG will look for methods annotated with @Test.  So back in the POM, we say:

    <plugin>
        <!--
            Run tests specified in the testng.xml file for integration-tests
        -->
        <groupId>org.apache.maven.plugins</groupId>
        <artifactId>maven-failsafe-plugin</artifactId>
        <version>2.5</version>

        <configuration>
            <suiteXmlFiles>
                <suiteXmlFile>src/test/it/testng.xml</suiteXmlFile>
            </suiteXmlFiles>
            <systemPropertyVariables>
                <seleniumUrl>${seleniumUrl}</seleniumUrl>
                <appUrl>${appUrl}</appUrl>
            </systemPropertyVariables>
        </configuration>

        <executions>
            <execution>
                <id>integration-test</id>
                <phase>integration-test</phase>
                <goals>
                    <goal>integration-test</goal>
                </goals>
            </execution>
        </executions>
    </plugin>
</plugins>
 

We bind it to the integration-test phase so it's run after Selenium starts. You'll notice that we pass seleniumUrl and appUrl into the TestNG tests. We pass those as parameters into a @BeforeSuite method; we'll touch on that in a moment.

Last bit, we're going to launch Xvfb, but only if we're on a UNIXy operating system, so we'll wrap it in a profile stanza:

<profiles>
    <profile>
        <id>xvfb-started</id>
        <activation>
            <os>
                <family>unix</family>
            </os>
        </activation>
        <build>
            <plugins>
                <plugin>
                    <!--
                        Before running the Selenium server, start a Xvfb to display
                        browsers into.
                    -->
                    <groupId>org.codehaus.mojo</groupId>
                    <artifactId>selenium-maven-plugin</artifactId>
                    <executions>
                        <execution>
                            <id>xvfb</id>
                            <!--
                                We want to start Xvfb *before* pre-integration-tests, but the
                                only phase before pre-integration-test is package, so we put it
                                there.
                            -->
                            <phase>package</phase>
                            <goals>
                                <goal>xvfb</goal>
                            </goals>
                            <configuration>
                                <display>${selenium.DISPLAY}</display>
                                <options>
                                    <option>${xvfb.option}</option>
                                </options>
                            </configuration>
                        </execution>
                    </executions>
                </plugin>
            </plugins>
        </build>
    </profile>
</profiles>

Oh the hack.  Ironically, Maven does not have a way to specify that a particular execution id depends on another execution id, so we had to bind launching Xvfb to the package phase. Maybe in Maven 3.0?

Getting into the actual code, earlier I mentioned the @BeforeSuite attribute, in that method we'll configure Selenium for our purposes:

.....
 
        /*
         * When running TestNG from within Eclipse, the plugin passes this value in to parameters it cannot substitute.
         */
        private static final String PARAMETER_NOT_FOUND = "not-found";
.....
        /*
         * Passing in these properties will allow you to use the specified values if the parameters are not found by TestNG.
         */
        public static final String SELENIUM_URL_PROPERTY = "selenium.url";
 
        public static final String APPLICATION_URL_PROPERTY = "app.url";
 
        private static WebDriver driver;
 
        private static WebDriverBackedSelenium selenium;
 
        private static String sessionString;
        private static String appUrl;
.....
        @BeforeSuite
        @Parameters( { "seleniumUrl", "appUrl" })
        public void startBrowser(String seleniumUrl, String appUrl) throws Exception {
                
                if (PARAMETER_NOT_FOUND.equals(seleniumUrl)) {
                        seleniumUrl = System.getProperty(SELENIUM_URL_PROPERTY);
                }
                if (PARAMETER_NOT_FOUND.equals(appUrl)) {
                        appUrl = System.getProperty(APPLICATION_URL_PROPERTY);
                }
 
                driver = new RemoteWebDriver(new URL(seleniumUrl), DesiredCapabilities.firefox());
 
                // This will cause all find-element operations to keep trying for up to 10 seconds automatically.
                driver.manage().timeouts().implicitlyWait(10, TimeUnit.SECONDS);
 
                selenium = new WebDriverBackedSelenium(driver, appUrl);
 
                /*
                 * Generate a unique string so if we update a field to a new value, we know it's going to be a new value (and can compare that it's so).
                 */
                sessionString = System.currentTimeMillis() + "";
 
                selenium.open(appUrl);
 
        }
....
        @AfterSuite
        public void stopBrowser() {
                driver.close();
 
        }
.....
 

This method is called before our test suite runs.  We do some extra work to allow developers to run TestNG with -Dselenium.url and -Dapp.url if running using the TestNG plugin for Eclipse to pass in parameters.  We also configure WebDriver to automatically wait 10 seconds for events to occur.  Don't forget to tell people writing tests that they don't need to do this again in their code!

This configuration gives us a handful of options, developers can boot up a Selenium server by issuing "mvn -Dselenium.background=false integration-test", then run tests either in Eclipse or by calling surefire directly with "mvn surefire:integration-test".

Reporting the News

To recap, we can start a container, deploy our application, wait for it to start, then launch Selenium, run some tests and shut Selenium down when our tests have finished.  Our final goal is to report an announce the results of our test. Generating reports is pretty easy, we'll just add the following to our POM:

<reporting>
    <plugins>
        <plugin>
            <groupId>org.apache.maven.plugins</groupId>
            <artifactId>maven-surefire-report-plugin</artifactId>
            <version>2.5</version>
            <reportSets>
                <reportSet>
                    <id>integration-tests</id>
                    <reports>
                        <report>report-only</report>
                    </reports>
                    <configuration>
                        <outputName>failsafe-report</outputName>
                        <reportsDirectories>
                            <reportsDirectory>${project.build.directory}/failsafe-reports</reportsDirectory>
                        </reportsDirectories>
                    </configuration>
                </reportSet>
            </reportSets>
        </plugin>
    </plugins>
</reporting>
 

Wow, that was easy.  In Hudson, we configure the selenium-tests project to run the following Maven Goals: integration-test site-deploy failsafe:verify. This asks Maven to perform the tests, deploy the site and then fail if the tests failed.

I've also configured Hudson to send an email out if this build fails, that email contains the URL directly to the Selenium Failsafe report!

Now we have an automated deployment mechanism, Selenium tests and reporting.

Awesome!

The checkout process is quite possibly the scariest part of the online shopping experience. Customers get anxious about divulging sensitive personal information, not to mention parting with their hard-earned cash. Long and complicated customer registration forms to fill out, strange error messages, security concerns, too many steps, etc., all contribute to shopper frustration and all too often, cart abandonment.

We recently did a review of our current out-of-the-box checkout process and we made the following observations:

• There are too many steps, resulting in a high cart abandonment rate.
• The distinction between customer types is confusing. "Am I a registered customer or an existing one?" "What's a 'guest'?"
• Giving the option to register during the first step of checkout is an unnecessary distraction. It gives customers too much time to change their mind!
• While the cart total is visible during each step, it doesn't provide a breakdown on specific items that have been added to the cart.
• Customers who have a promotion code don't know where to enter it, which leads to frustrated calls to customer service.

We started looking at ways to simplify the process and we came up with a shorter, smoother, and clearer checkout process that works well for various industries and customer types/profiles. In this new and improved checkout process, there are only three stages:

• shipping information
• billing information
• receipt and (optional) registration.
  

Shipping Information

In the new process, the login and shipping information pages are merged into a single page. If a returning customer can't remember their login name or password, they can skip it and just enter the shipping details without any extra clicks. Here, guests and new customers are treated the same.

The contents of the shopping cart are displayed on this page and on both of the other checkout pages, so the customer always knows exactly what they're buying. The coupon and promo code fields are also available at each step, under the shopping cart summary. Once a coupon or promo code is applied, the cart total is updated automatically via AJAX.

Billing Information

The billing step asks for the customer's email address and any other information required for invoice purposes. All the shipping information entered in the last step appears once again for final customer review. The final confirmation block is a good place to have a subscription or "sign up for email alerts" check box. Even if customers don't create an account, they may still choose to be on the mailing list.

Receipt and Registration

Unlike the standard checkout process, the customer is only asked to register for an account after they've made their purchase. Some retailers might worry that nobody will register unless they're required to. If that's a concern, you can provide some incentive for registering (like a discount on their next purchase). Keep in mind that the point is to avoid distracting the shopper from their goal, which is to make their purchase.

If the customer registers for an account after checkout is complete, that order will be associated with the new account. Unfortunately, previous orders they created will not be, even if they were created under the same email address as the new account. (This is for security reasons.)

This new checkout process will require using AJAX/JavaScript validation at every step, making it a bit tedious to code, but worth it in the long run.

We are currently in the process of integrating this new process for several of our clients and will be keeping a close eye on their performance and results. Of course, this process may not work for every ecommerce site. Hopefully this will at least give you some inspiration for ways to improve your own checkout process. There are always areas that can be simplified and streamlined. Just keep testing various aspects of your checkout process. That's the only way to really learn what works best for your audience.

Stay tuned and subscribe to our RSS feed because down the road we will follow up with another post regards the results and observations.

In addition to manual test cases and unit tests, Elastic Path relies on FIT (Framework for Integrated Test). FIT allows testers to write tests without writing any code. FIT is a good framework for use with EP because we usually have a set of defined business rules to which we know the expected results and outcomes.

What is FIT?

Originally developed by Ward Cunningham, FIT is based on JUnit. Unlike JUnit, which focuses on the functionality of one method, FIT is designed for testing components. For example, a typical JUnit test might check the calculation of shipping costs, whereas a FIT test would check the functionality of the entire checkout process, including selecting a customer, adding a product to the shopping cart, calculating the shipping costs, and completing the checkout process.

What does a FIT test look like?

A FIT test consists of a set of HTML tables that perform various tasks. This generally includes:

• initializing the data required for the test
• performing some actions
• checking the results of those actions.

The following is an example of a FIT test:

The test should always begin with a description of what the test is trying to accomplish. In this example, we are testing to see if adjusting inventory for a particular product works as expected.

The first table after the description points to a fixture. The fixture is a Java class that provides methods for use in the test. Every application that uses the FIT framework for testing has its own set of fixtures that are unique to its implementation. For example, the fixture that the test above references is PaymentServiceFixture. Therefore, we need a Java class named PaymentServiceFixture. Indeed, there is a class that exists with that name.

So where is all the data? That's in the next table:

When the test is run, this table tells the FIT framework to call the useScenario(String) method in PaymentServiceFixture or if not found there, in a superclass. In this example, the method resides in EpDoFixture, which is a superclass of PaymentServiceFixture.

How does FIT know to call a method named useScenario() that's expecting one parameter? It concatenates all the words in the odd cells in the table, removes the spaces, can captializes the first letter of each word after the first one. Therefore, in the example above, it is looking for a useScenario() method with one parameter inside it. (If the first cell of the method contains the keyword "check", the behavior is a bit different; it concatenates the contents of the even cells and tries to look for a getter method with that name.)

The useScenario(String) is a custom method in Elastic Path Commerce that attempts to parse the specified .scn file. The file actually contains another FIT test. We use these scenario FIT tests to help populate the data required to perform other tests. If we look in CA_Store.scn, we can see that there are mostly tables that aid in the set up of a store.

The next step after adding the common required data is to add the test-specific data. We should be adding inventories to the store. However, looking through the code in PaymentServiceFixture.java would not reveal to us that this method is available to use, since it is not one of the methods within that class. However, there is a method in CatalogSetUpFixture.java that does just that. We have another custom method in Elastic Path that allows us to call an operation in another fixture. We use the operation by the keyword "use", followed by the name of the fixture in the next cell. After we declare this in the first row of a table, we can then use methods of that class. Note that you can call any methods of the class as long as you have declared the class to use in the first row of the table.

The next step is to call the method addInventories(). Now imagine we have no idea what fields we need to populate this method. Let's go to the code to see what we need. Here is the code from CatalogSetUpFixture.java:

     /**
      * Action method <code>add inventories</code>.
      * 
      * @return <code>AddInventories</code> SetUpFixture
      */
     public SetUpFixture addInventories() {
          return new AddInventories();
     }

Because we know that addInventories() does not take in any parameters, all we need on the second row is just one cell, namely "add inventories". Here is what the table looks like at this point:

Still with me? Good. This part is going to get a little confusing, so pay attention. The addInventories() method returns a class of type Fixture, we can infer that there will be more rows in this table. But how do the row of tables look like? In order to answer this question, we have to go to the class AddInventories. In this class, we get the variables through a special method, whose name is a concatenation of the parameters it is expecting. As you can see from the following code, this looks pretty strange.

/**
      * SetUpFixture to add inventories.
      */
     public class AddInventories extends SetUpFixture {
          /**
           * Action method <code>product sku code warehouse code qty on hand reserved qty allocated qty</code>.
           * 
           * @param skuCode sku code
           * @param warehouseCode warehouse code
           * @param qtyOnHand qty on hand
           * @param reservedQty reserved qty
           * @param allocatedQty allocated qty
           */
          public void productSkuCodeWarehouseCodeQtyOnHandReservedQtyAllocatedQty(final String skuCode, final String warehouseCode,
                    final int qtyOnHand, final int reservedQty, final int allocatedQty) {
               Warehouse warehouse = warehouseService.findByCode(warehouseCode);
               catalogPersister.persistInventory(skuCode, warehouse, qtyOnHand, reservedQty, allocatedQty);
          }
 
          /**
           * Action method <code>product sku code warehouse code qty on hand reserved qty allocated qty reorder minimum reorder qty</code>.
           * 
           * @param skuCode sku code
           * @param warehouseCode warehouse code
           * @param qtyOnHand qty on hand
           * @param reservedQty reserved qty
           * @param allocatedQty allocated qty
           * @param reorderMinimum reorder minimum
           * @param reorderQty reorder qty
           */
          public void productSkuCodeWarehouseCodeQtyOnHandReservedQtyAllocatedQtyReorderMinimumReorderQty(final String skuCode,
                    final String warehouseCode, final int qtyOnHand, final int reservedQty, final int allocatedQty, final int reorderMinimum,
                    final int reorderQty) {
               Warehouse warehouse = warehouseService.findByCode(warehouseCode);
               catalogPersister.persistInventory(skuCode, warehouse, qtyOnHand, reservedQty, allocatedQty, reorderMinimum, reorderQty);
          }
     }

For this part, we need to separate that long method name into variable names. The order of the names must be the same as the order they appear in the method name. After this, the table should look like the following:

After we have all the variables split in to the correct format, we can add the data. We can add as many data rows as we want. Finally, table should look similar to the following:

The convention here is to color variables blue so that they stand out. The color properties are ignored by the FIT framework, so it is encouraged that you use colors for variables.

Validation

Now that we have added all the data neccessary for this test, we have to test whether or not the operation was successful. Here is the following table that does the trick.

Time for a quick quiz. What is the expected method name that we should see in PaymentServiceFixture.java? If you guessed inventorySummaryProductSkuCodeWarehouseCode(String, String), you're absolutely correct! This is the code in PaymentServiceFixture.java:

     /**
      * Checks inventory summary information.
      * 
      * @param skuCode product SKU code
      * @param warehouseCode warehouse code
      * @return <code>ParamRowFixture</code> parameterized with <code>InventoryDetailsRow</code> objects
      * @see <code>InventoryDetailsRow</code>
      */
     public Fixture inventorySummaryProductSkuCodeWarehouseCode(final String skuCode, final String warehouseCode) {
          ProductSku productSku = productSkuService.findBySkuCode(skuCode);
          Warehouse warehouse = warehouseService.findByCode(warehouseCode);
          Inventory inventory = productSku.getInventory(warehouse.getUidPk());
          return new ParamRowFixture(new Object[] { new InventoryDetailsRow(inventory) });
     }

The method returns a Fixture, which means another row in the table is required. But what does the next row look like? To find out, we need to look at the code into InventoryDetailsRow. We wrap the information into a row object. This is required by the FIT framework. Let's take a look at InventoryDetailsRow.java:

package com.elasticpath.fit.rowdata;
 
import com.elasticpath.domain.catalog.Inventory;
 
/**
 * Used in FIT tests as row representation of inventory details.
 */
public class InventoryDetailsRow {
 
     /**
      * Quantity on hand.
      * FIT field: qty on hand.
      */
     public int qtyOnHand;
 
     /**
      * Reserved quantity.
      * FIT field: reserved qty.
      */
     public int reservedQty;
 
     /** allocated quantity. */
     public int allocatedQty;
 
     /** available qty in stock. */
     public int availableQtyInStock;
 
     /**
      * Constructor fills the inventory details.
      * 
      * @param inventory the inventory
      */
     public InventoryDetailsRow(final Inventory inventory) {
          qtyOnHand = inventory.getQuantityOnHand();
          reservedQty = inventory.getReservedQuantity();
          allocatedQty = inventory.getAllocatedQuantity();
          availableQtyInStock = inventory.getAvailableQuantityInStock();
     }
}

The public fields in this class are the fields that we need in the FIT tables. They will determine the column names for the next row. In this case there are 4 fields: qtyOnHand, reservedQty, allocatedQty, and availableQtyInStock. We then add another row, which will contain the values that we expect the test to have. These fields also follow the concatenation convention, so you can either use the exact name of the field, or put spaces between each of the uppercase characters and use a lowercase letter instead. Notice that the backend for creating data and checking data is a bit different, even though in our FIT tables, it looks similar. This is because when we are checking data, FIT needs to validate the values in the table against the system, and it does so with row objects in this case. Finally, our last table looks like this:

If we run the test, this is what we will see in our results page:

Click on picture to enlarge

FIT is an invaluable testing tool for ensuring code quality and stability for the Product Development team. In addition to providing us with component tests, it ensures a measure of stability with our daily builds. After every build, our server runs all FIT tests in the system to make sure recent code commits don't cause any of the tests to fail. In that way, it acts as a source for regression testing. We've also recently started using FIT for acceptance testing. Since testers don't need to write code, they can write the tests (tables) in advance. Developers would then implement the fixture code after they have developed a new feature. A new feature is not complete until it passes all the FIT tests in that area. Finally, we are realizing other uses for FIT, including:

• Sample data population (using scenario data to build sample data, including catalogs, products, customers)
• Performance testing (creating batches of data and running the application under JProfiler to measure memory usage, CPU time, etc.)

Stay tuned for more detailed explanation on these two topics in another blog post!

Selenium is a suite of free, community-developed tools for automated testing of web applications. It provides test recording and playback, and supports test playback in both Firefox and Internet Explorer. For our testing purposes, we use two tools in the suite: Selenium IDE and Selenium Core.

Selenium IDE is a Firefox extension that lets you record, edit, and debug Selenium tests. To start it, launch Firefox and choose Selenium IDE in the Tools menu. In the IDE window, you can create a test suite and record all the test cases you want to make, then save them to your hard drive in the file format of your choice. HTML, Java, C#, Perl, PHP, Python and Ruby are all supported formats, but we save the test cases as HTML so we can re-use them in our IE test suite (more on this later).

Selenium Core can do a lot of things, but we're only interested in the TestRunner tool, which we use to run our HTML test cases in Internet Explorer. To run the test cases, you need to create a test suite file in an HTML editor and add links to the HTML test case files created in IDE. To see how to do this, you can look at the example test suite files included in Selenium Core.

After that's done, run TestRunner by double-clicking TestRunner.HTA in the Core Dir\Core folder where you unzipped the Selenium Core package. Then enter the path and name of the test suite you want to run in the Test Suite box and click Go.

Real-World Selenium

For one large customer project, we run test cases against four different online stores in three difference locales. For each store, we've grouped the test cases into a test suite file. This test suite file is the master test case list and can be loaded into either Selenium IDE or TestRunner,depending which browser needs to be tested.

Installing Selenium IDE

1. In Firefox, go to http://seleniumhq.org/.

2. Click the Download tab.

3. Download the Selenium IDE.

4. Click Install Now. This will automatically install Selenium IDE in Firefox. You will need to restart Firefox to use IDE.

Installing Selenium Core

1. Go to http://seleniumhq.org/.

2. Click the Download tab.

3. Download the Selenium Core package to your local system.

4. Unzip the Core package. TestRunner application is located in the folder where you unzipped Core, under Core Dir\Core.