Technical Blog

7 Posts tagged with the performance tag

The performance of the OOTB CachingStoreResolver begins to degrade when more than a handful of stores are configured in EP. This post describes a simple replacement implementation that scales much better.

 

The problem

 

The StoreResolver is used by the storefront to resolve stores by either store code or domain. The OOTB CachingStoreResolver uses a SimpleTimeoutCache to save store codes and domain-to-store mappings. The default cache timeout is 1 minute. When an entry is not found in the cache, the request is delegated to the StoreResolver to resolve the request from the database and the result is cached if valid. All this looks pretty reasonable on the surface.

 

The scalability problems are caused by how the cache is populated. If a store code or domain are not found in cache, then the StoreResolver reads ALL stores from the database and then returns only the matching store code. If there are 100 stores and the default cache timeout is used then we can expect cache entries to expire nearly twice every second. While this is not ideal, there wouldn't be much of an issue if reading a store were a lightweight query. However OpenJPA brings along a large number of related objects and the operation ends up being quite expensive.

 

It is also worth noting that a request with an invalid store code or domain will always have a cache miss and will result in fetching all stores from the database.

 

Two solutions

 

The simplest approach is to increase the cache timeout, but this does not fix the issue of handling invalid store codes or domains.

 

A more complete solution is to replace the CachingStoreResolver with the following class which caches all store codes or domain-to-store mappings with a single query.

 

import java.util.Collection;
import java.util.Map;

import com.elasticpath.sfweb.util.impl.StoreResolverImpl;

 

/**
* This re-implements the OOTB CachingStoreResolver to cache all stores in a single database query.
*/
public class ReplacementCachingStoreResolverImpl extends StoreResolverImpl {

 

    /**
     * Get the set of valid store codes.
     *
     * @return a collection of all codes for complete stores.
     */
    @Override
    protected Collection<String> getStoreCodes() {
        final String cacheKey = "storeCodes";
        Collection<String> storeCodes = (Collection<String>) getFromCache(cacheKey);
        if (storeCodes == null) {
            storeCodes = super.getStoreCodes();
            addToCache(cacheKey, storeCodes);
        }
        return storeCodes;
    }

 

    /**
     * Creates a map with all the store domains mapped to their store codes.
     *
     * @return a map populated with all store domains mapped to
     *            their corresponding store codes.
     */
    @Override
    protected Map<String, String> getDomainToStoreMapping() {
        final String cacheKey = "domainToStoreMap";
        Map<String, String> storeCodeMap = (Map<String, String>) getFromCache(cacheKey);
        if (storeCodeMap == null) {
            storeCodeMap = super.getDomainToStoreMapping();
            addToCache(cacheKey, storeCodeMap);
        }
        return storeCodeMap;
    }

 

    private Object getFromCache(final String cacheKey) {
        // TODO - Hook in your cache of choice, could be a SimpleTimeoutCache
        return null;
    }
   
    private void addToCache(final String cacheKey, final Object obj) {
        // TODO - Hook in your cache of choice, could be a SimpleTimeoutCache
    }

 

}

3 Comments Permalink

Recently, I needed to reproduce a problem that was only occurring when the storefront was under heavy load. As a developer, I cannot easily generate realistic load in my environment and I don't have access to a live production system, so I used JMeter to help me out. JMeter is a Java application for performing load tests on web applications. In this post, I'll show how to use JMeter to quickly set up a simple performance test against the Elastic Path Commerce storefront.

 

I wanted to see how long it would take to perform a search in the storefront for Canon products. In this case, I wanted JMeter to fire the following query a few hundred times using one or more threads: http://demo.elasticpath.com:8080/storefront/search.ep?categoryId=&keyWords=canon&submit=search

 

First, I created a thread group under the test plan tree node. (Right-click the test plan and choose Add->Thread Group.) If you want to simulate more than one user hitting the same search at the same time, you can set the number of threads in the thread group configuration.

img1_1.png

 

Next, I added a loop controller to my thread group. (Right-click the thread group and click Add->Logic Controller->Loop Controller.) Here, I defined how many times the loop will run. In my case, I checked the Infinite checkbox.

img2.png

 

Then, I added an HTTP request to the loop controller. (Right-clicking the loop controller and click Add->Sampler->HTTP Request.) I set the Server Name, Server Port, URL Path, chose GET in the dropdown and set three parameters in the request URL.

img3_1.png

 

At this point, I started the test by clicking Run->Start. JMeter starts to execute the HTTP request I defined in the loop controller. Since I set the loop count to Forever, it will continue to execute until I stop it.

 

Now, it would be a good idea to attach some reports to debug the execution. I tried a few reports and found that the View Results Tree presented all the information I needed. It shows the raw request sent by JMeter and the response data the server is providing. You can examine the server response to make sure the HTTP request you configured is doing what it is expected to do.

 

To add a View Results Tree report to your test plan, right-click your HTTP request test, and choose Add->Listener->View Results Tree.

You can finally see the results of each call on the report.

 

img4.png

 

You can build more complex tests, with several simultaneous requests over different URLs. For example, you can configure JMeter to first simulate a user logging in to your application (and hold session information), then a few requests to simulate the user navigating through the web application and then repeat this test a few hundred times using as many threads as you want (or your machine can handle).

 

Using JMeter, I was able to simulate a sufficently heavy load to reproduce the problem I wanted to reproduce and this post should help you do the same thing if you face a similar problem. For more details on JMeter, go to: http://jakarta.apache.org/jmeter/

 

Hope you enjoy!

2 Comments Permalink

If some of the recent performance-related posts by our much cherished Get Elastic blogger extraordinaire Linda are starting to worry you, fear not! With the newly released Elastic Path 6.2, the Product Development and Performance teams at Elastic Path have done a fantastic job ramping up the standard performance of the product. I'll let them brag about the numbers at a later date, but today's post is to dig into the guts of the caching introduced into the system, and how you can start tweaking some of the configuration settings to squeek out every millisecond for page responses.

 

Within Elastic Path, there are now two caches that are available to be tweaked:

  • Application-level: Sitting between the view layer and the data access layer
  • Persistence-level: A Level-2 cache, within the OpenJPA ORM framework

 

Application Level Cache

All products loaded within the Storefront application via the StoreProductService will be from an Ehcache-backed cache by default. Each application is responsible for loading products via a ProductRetrieveStrategy. You'll notice storefront has two new configurations to facilitate using Ehcache:

 

Cache.xml:
     <bean id="productCache" class="org.springframework.cache.ehcache.EhCacheFactoryBean">
          <property name="timeToLive" value="600"/>
          <property name="timeToIdle" value="600"/>
     </bean>

     <bean id="cachingProductRetrieveStrategy" class="com.elasticpath.sfweb.service.impl.EhCacheProductRetrieveStrategyImpl">
          <property name="productService" ref="productService" />
          <property name="cache" ref="productCache" />
     </bean>

 

ServiceSF.xml:
    <alias name="cachedSettingsReader" alias="settingsReader"/>
    <alias name="cachingProductRetrieveStrategy" alias="productRetrieveStrategy"/>

 

You'll note that the storefront is now using aliases in the Spring configuration to override the same bean definitions in the default core service.xml. This allows the storefront to setup caching-specific classes. For tweaking purposes, the productCache bean definition should be updated to optimize the Ehcache settings. By default, Spring's EhCacheFactoryBean will initialize the cache to allow overflow to disk, use LRU eviction and limit the in-memory size to 10k objects. For catalogs with a larger product mix, these settings should be optimized and potentially moved to an distributed cache via Terracotta to keep the JVM heap size to a reasonable level.

 

You can choose to add new properties here to tweak these values, or add an ehcache.xml configuration file to the classpath and define a specific cache name as part of the productCache definition to use. A quick tip on monitoring the cache statistics for tweaking the settings during load test is to setup JMX monitoring for the storefront cache. This can be done via two steps:

 

Running the appserver with remote JMX enabled (no authentication for a non-production environment):

-Dcom.sun.management.jmxremote -Dcom.sun.management.jmxremote.port=6969 -Dcom.sun.management.jmxremote.ssl=false -Dcom.sun.management.jmxremote.authenticate=false

 

Configuring the JMX beans for Ehcache in cache.xml. Note in this case we are explicitly setting up a cache manager, which we can also use to specify a custom Ehcache configuration file instead of the default ehcache.xml:

<?xml version="1.0" encoding="UTF-8"?>
<beans xmlns="http://www.springframework.org/schema/beans" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans-2.5.xsd ">

    <bean id="productCache" class="org.springframework.cache.ehcache.EhCacheFactoryBean">
        <property name="timeToLive" value="600"/>
        <property name="timeToIdle" value="600"/>
        <property name="cacheManager" ref="cacheManager" />
    </bean>

    <bean id="cachingProductRetrieveStrategy" class="com.elasticpath.sfweb.service.impl.EhCacheProductRetrieveStrategyImpl">
        <property name="productService" ref="productService" />
        <property name="cache" ref="productCache" />
    </bean>

    <bean id="cacheManager"
        class="org.springframework.cache.ehcache.EhCacheManagerFactoryBean">
    </bean>

    <!-- Spring initialization of ehCache's mbeans -->
    <bean id="ehCacheMBeanRegistration"
        class="org.springframework.beans.factory.config.MethodInvokingFactoryBean">
        <property name="staticMethod"
            value="net.sf.ehcache.management.ManagementService.registerMBeans" />
        <property name="arguments">
            <list>
                <ref bean="cacheManager" />
                <ref bean="mbeanServer" />
                <value>true</value>
                <value>true</value>
                <value>true</value>
                <value>true</value>
            </list>
        </property>
    </bean>

    <bean id="mbeanServer" class="org.springframework.jmx.support.MBeanServerFactoryBean">
        <property name="locateExistingServerIfPossible" value="true" />
    </bean>

</beans>

 

Connecting up with jConsole lets us check the cache settings are properly configured, and to check the statistics, as per the below image. We should see updates to statistics as you browse the storefront and load more products:

screen-capture-1.png

 

 

Persistence Level Cache

As part of the Elastic Path 6.2 release, the included OpenJPA library has been upgraded to a 1.2.1 version, which overcomes some data cache issues in the previous 1.0.1 version. Please see the OpenJPA 1.2.1 documentation for all details on the native OpenJPA datacache. We'll go over some of the changes that enable the data cache using the native implementation.

 

Annotations:

All transactional persistent entities (anything submitted or updated regularly as an online transaction such as orders, payments and customers) have a new data cache annotation so that they are non-cacheable. By default, entities are enabled in the data cache unless this annotation is present and explicitly disabled the caching. All relatively static data, such as catalog entities should be part of the cache and thus will be missing this new annotation.

 

@DataCache(enabled = false)

 

Persistence.xml Configuration:

As part of the persistence unit configuration, three new properties are configured by default:

            <property name="openjpa.DataCache" value="true"/>
            <property name="openjpa.RemoteCommitProvider" value="sjvm"/>
            <property name="openjpa.DataCacheTimeout" value="1000"/>

 

These values are used for:

  • openjpa.DataCache - enabling the cache, and specifying the cache properties. We recommend tweaking this value to accomodate the cache size according to the size of the underlying data: (true(CacheSize=25000, SoftReferenceSize=0))
  • openjpa.RemoteCommitProvider - specifying the commit provider. For a clustered setup, evictions should be handled via configuring this setting to use JMS or TCP-based evictions.
  • openjpa.DataCacheTimeout - maximum time to live for entities in the cache

 

Also suggested is to tune the query cache, which is enabled with default values when the DataCache property is set to true:

<property name="openjpa.DataCache" value="true(CacheSize=25000, SoftReferenceSize=0)"/>
<property name="openjpa.RemoteCommitProvider" value="sjvm"/>
<property name="openjpa.QueryCache" value="CacheSize=25000, SoftReferenceSize=0"/>

 

Tip: Catching Cache Hits and Misses

If caching is enabled and you're still seeing a large amount of database queries from the storefront server, you can enable data cache Log4j tracing and grep out the hits and misses logged to track down which entities and queries are mistakingly hitting the database. Most of the times, these can be tracked down to accidental evictions or entities within the inheritance hierachy being disabled from cache.

 

log4j.category.openjpa.DataCache=TRACE

 

This spits out some logs similar to below that's easily greppable to count and track which entities are creating problems:

DEBUG 2009-09-10 12:56:33,918 org.apache.renamed.openjpa.lib.log.CommonsLogFactory$LogAdapter.trace(CommonsLogFactory.java:76) - Cache hit while looking up key "com.elasticpath.domain.attribute.impl.ProductTypeProductAttributeImpl-1".
DEBUG 2009-09-10 12:56:33,926 org.apache.renamed.openjpa.lib.log.CommonsLogFactory$LogAdapter.trace(CommonsLogFactory.java:76) - Cache miss while looking up key "org.apache.renamed.openjpa.datacache.QueryKey@b7ee9cab[query:[SELECT ps.skuCodeInternal FROM ProductSkuImpl ps

 

OpenJPA Cache Plugins

The native data cache within OpenJPA is architected to be swappable via plugin configuration. This allows the ability to swap in alternative caching technologies like Ehcache or Oracle Coherence to support extensive scalability requirements, as both Ehcache/Terracotta and Oracle Coherence support distributed caching setups. We've tested Ehcache and Coherence plugins internally with favourable results.

 

An Ehcache plugin is provided by the Ehcache group, however this version must be repackaged to match the custom package names of the Elastic Path specific org.apache.renamed.openjpa jar. Once a repackaged instance of the Ehcache-OpenJPA jar is in the classpath, the configuration changes to:

<property name="openjpa.DataCacheManager" value="ehcache"/>

 

Similarily enough, OpenJPA commiter Pinaki has posted the initial workings of an Oracle Coherence plugin on his blog, along with some additional JPA caching insights. Elastic Path with OpenJPA and Oracle Coherence is probably worth an entire blog entry in itself (coming soon!), but the same configuration settings apply, along with the need to repackage the code to point at the Elastic Path OpenJPA jar:

 

<property name="openjpa.DataCacheManager" value="coherence"/>

 

So there it is, two new caching mechanisms to tweak and fine-tune as part of load tests that should favourably reflect storefront page load times, and hopefully conversion rates and green dollar signs. Feel free to ask away about some of our load testing and tuning war stories. We'd be happy to talk about our hands on experience with the recent caching work.

6 Comments Permalink


The performance team at Elastic Path has recently gone through Oracle RAC validation with Elastic Path 6.1.1 and made it out the other side unscathed. And the best part, is that there are no code changes required on EP out-of-the-box to fully support Oracle RAC with Fast-Connection-Failover (FCF).


The benefits of Oracle RAC (Real Application Clusters, or Oracle database clustering in simple terms) are three-fold: performance, scalability, and reliability. Which one matters the most to you depends on your needs, but usually having the assurance of database failover is the most valuable, with scalability and performance coming a close second.


In our testing, we used WebLogic 10.0.1 and Oracle 11g (Release 1) on physical machines using Intel Xeon quad-core 2.5Ghz CPUs and 8GBs RAM. The OS was 64-bit RedHat EL 5.  In-house, we are typically able to push a single Oracle node to capacity with 3 EP storefront nodes. For our validation testing, with a four storefront configuration, we were utilizing roughly 50-60% capacity on a two node RAC configuration. The following is a rough guide for setting up RAC for EP.

 

RAC Configuration w/ EP

Deployment and configuration of Oracle Clusterware and Oracle 11g was fairly straight-forward and required no special configuration with Elastic Path, other than the standard RAC connection config outlined below. Oracle's online documentation for the Clusterware  set up is excellent and very detailed when you need to drill down.


Once the Clusterware and database are up and running, and your data has been populated, there are many different ways to set up RAC with WebLogic. See the WebLogic documentation for details. WebLogic recommends the use of multi data sources to connect to the RAC nodes. This method supports failover and load-balancing at the application level which is more effective as WebLogic's health monitors can be used and failover is done more quickly than Connect-Time failover or allowing the cluster-ware to handle this. It is recommended to set up a data source for each RAC node. Below is a configuration example for the data source XML; it is based on a two node setup (a data source for each node) and the DS pool.



WebLogic Data Source Example XML

<jdbc-data-source>

<name>jdbcPool</name>

<jdbc-driver-params>

  <url>jdbc:oracle:thin:@lcqsol24:1521:snrac1</url>

  <driver-name>oracle.jdbc.OracleDriver</driver-name>

  <properties>

   <property>

    <name>user</name>

    <value>wlsqa</value>

   </property>

  </properties>

  <password-encrypted>{3DES}aP/xScCS8uI=</password-encrypted>

</jdbc-driver-params>

<jdbc-connection-pool-params>

  <test-connections-on-reserve>true</test-connections-on-reserve>

  <test-table-name>SQL SELECT 1 FROM DUAL</test-table-name>

</jdbc-connection-pool-params>

<jdbc-data-source-params>

  <jndi-name>jdbcDataSource</jndi-name>

</jdbc-data-source-params>

</jdbc-data-source>


<jdbc-data-source>

<name>jdbcPool2</name>

<jdbc-driver-params>

  <url>jdbc:oracle:thin:@lcqsol25:1521:SNRAC2</url>

  <driver-name>oracle.jdbc.OracleDriver</driver-name>

  <properties>

   <property>

    <name>user</name>

    <value>wlsqa</value>

   </property>

  </properties>

  <password-encrypted>{3DES}aP/xScCS8uI=</password-encrypted>

</jdbc-driver-params>

<jdbc-connection-pool-params>

  <test-connections-on-reserve>true</test-connections-on-reserve>

  <test-table-name>SQL SELECT 1 FROM DUAL</test-table-name>

</jdbc-connection-pool-params>

<jdbc-data-source-params>

  <jndi-name>jdbcDataSource2</jndi-name>

  <global-transactions-protocol>OnePhaseCommit</global-transactions-protocol>

</jdbc-data-source-params>

</jdbc-data-source>


<jdbc-data-source>

<name>jdbcNonXAMultiPool</name>

<jdbc-data-source-params>

  <jndi-name>jdbcDataSource</jndi-name>

  <algorithm-type>Failover</algorithm-type>

  <data-source-list>jdbcPool,jdbcPool2</data-source-list>

  <failover-request-if-busy>true</failover-request-if-busy>

</jdbc-data-source-params>

</jdbc-data-source>


 

Fast-Connection-Failover

WebLogic also supports Fast-Connection-Failover (FCF). This mechanism provides a means to receive event notification from the Oracle RAC nodes such as notification and cleanup of invalid connections, load balancing events, and node failures. In order to enable FCF, you must tweak the Oracle JDBC driver and add a couple additional properties to the data source connection such that it knows how to receive the ONS (Oracle Notification System) messages.

 

To enable FCF on a data source:

  1. In the WebLogic console, under the data source:
    1. In Driver Class Name, set the driver class to oracle.jdbc.pool.OracleDataSource.
    2. In Properties, set the ONS configuration string to subscribe to RAC's ONS messages, for example: ONSConfiguration=nodes=hostname1:port1,hostname2:port2
  2. Finally, make sure that ONS is properly configured on the RAC nodes and you have no blocking firewalls on those ports on either the RAC nodes or the application server nodes.
3 Comments Permalink

We had a client who needed to display entire product categories (several hundred products in all) in a set of drop down list boxes on their Elastic Path Commerce storefront homepage. The data that had to be loaded was quite minimal, just the product codes, display names and a few localized attributes.

 

In this type of situation, using the default product loader to retrieve this information would not be the best approach. This is due to the fact that each Product domain object contains a large amount of data such as prices, skus, attributes, inventory and recommendations etc.  Loading all these details requires a significant number of queries to be run on the database so applying this approach to a homepage with hundreds of products would result in extremely poor performance.

 

For this customer, a better solution was to create a set of lightweight product display classes, and have these mapped directly to some custom JPA native queries. These queries were tailored to return only the specific product details needed and therefore avoid loading any unnecessary information.  To further reduce the number of database trips, we also introduced a timed cache in the storefront controllers which would store frequently accessed catalog items for a set period of time before being refreshed. 

 

The combination of these two techniques reduced their page response times from tens of seconds (using the default product loader) to under a few seconds even in the worst case scenario where a stale cache had to be refreshed.   Since most of the time the information would be available in the cache, the amount of database overhead was kept to a minimum.

 

If your storefront scenario has similar requirements, then the following code examples may also be useful for your project.

 

Using Native SQL Queries with JPA

To get all product codes and product names in a particular category, you can use the EntityManager's createNativeQuery method to create a native query with a WHERE clause that passes in the language string and a specific category UID.

 

final String productNamesByCatSql =

"SELECT tp.code, tpldf.display_name AS displayName"

+ " FROM tproduct AS tp"

+ " INNER JOIN tproductldf AS tpldf"

+ " ON tpldf.product_uid = tp.uidpk"

+ " INNER JOIN tproductcategory AS tpc"

+ " ON tp.uidpk = tpc.product_uid"

+ " WHERE tpldf.LOCALE = ?1"

+ " AND tpc.category_uid = ?2";

 

long categoryUid = getCategoryUid(request);

long start = System.currentTimeMillis();

 

// create a native SQL query

final Query query = entityManager.createNativeQuery(productNamesByCatSql, ProductDisplayBeanImpl.class);

 

// retrieve a simplified product list for a given category

final List<ProductDisplayBean> custProducts = (List<ProductDisplayBean>)

     query.setParameter(1, shoppingCart.getLocale().getLanguage())

     .setParameter(2, new Long(categoryUid))

     .getResultList();

 

if (LOG.isDebugEnabled()) {

     long elapsedTimeMillis = System.currentTimeMillis() - start;

     LOG.debug("No. of products returned = " + custProducts.size() + ", elapsed time (ms) = " + elapsedTimeMillis);

     for (ProductDisplayBean productBean : custProducts) {

          LOG.debug("[" + productBean.getCode() + ":" + productBean.getDisplayName() + "]");

     }

}

 

Like JPQL, native queries can be named for easy reuse using the @NamedNativeQuery annotation, however it's probably a better idea to externalize all the native query strings into an *orm.xml file so that the SQL can be updated without having to change the class files.  If you do that, you'll have to change all the createNativeQuery()calls to createNamedQuery().

 

A very simple display bean containing only the product name and code is needed just to pass the information to the storefront.

 

 

 

public interface ProductDisplayBean {

 

     public String getCode();

     public void setCode(String code);

     public String getDisplayName();

     public void setDisplayName(String displayName);

}

 

public class ProductDisplayBeanImpl implements ProductDisplayBean {

 

     public static final long serialVersionUID = 5000000001L;

     private String code;

     private String displayName;

 

     public String getCode() {

          return code;

     }

 

     public void setCode(String code) {

          this.code = code;

     }

     public String getDisplayName() {

          return displayName;

     }

     public void setDisplayName(String displayName) {

          this.displayName = displayName;

     }

}

Caching Frequently Accessed Items

 

Once you've retrieved a set of items from the catalog, you can easily cache these results at the application layer using either a third party solution, such as Ehcache (http://ehcache.sourceforge.net), or your own caching mechanism.  Elastic Path 6.1 already provides some out-of-the-box caching classes that you can use in your own code.  We normally use our SimpleTimeoutCache class to hold a list of frequently retrieved products that don't need to be updated very often. The following code sample shows how you could integrate SimpleTimeoutCache with your product retrieval service class.

 

private static final long CACHE_TIMEOUT = 30000;  // cache expires after 30 seconds

private final SimpleTimeoutCache<String, List<Product>> productsCache = new SimpleTimeoutCache<String, List<Product>>(CACHE_TIMEOUT);

...

String storeCode = shoppingCart.getStore().getCode();

List<StoreProduct> products = productsCache.get(storeCode);

 

// if the store code does not exist in the cache, then get it from the database

// and update the cache

if (products == null) {

     final IndexSearchResult productResults = retrieveProducts(browsingRequest, category);

     List<Long> productUids = getProductsUsingPageNumber(pageNumber, storeCode, productResults);

     products = getProductRetrieveStrategy().retrieveProducts(productUids,

     shoppingCart, productLoadTuner);

     productsCache.put(storeCode, products);

}

 

The timeout value can be adjusted for your requirements. A larger value will reduce the number of trips to the database but the trade-off is a longer wait time for catalog product changes to appear.

4 Comments Permalink

Performance testing is a task often left to the last stages of software development, after the code is checked in, passed QA for functional testing, and no longer fresh in the minds of developers who wrote it. With JUnitPerf, you can leverage your existing suite of JUnit tests to get performance measurements while you're developing. The framework provides easy to use decorator classes for creating:

 

  • Timed tests to measure execution time
  • Load tests for measuring how your code runs with multiple requests

 

Performance tuning should always be done with a performance goal in mind, and JUnitPerf shouldn't be the end all measurement tool for the performance your code. Instead it is more useful for

 

  • Getting a basic idea of where your execution bottlenecks may lay
  • Evaluating the performance difference between multiple technical solutions
  • Uncovering immediate concurrency issues

 

JUnitPerf uses decorator classes to wrap existing JUnit tests. This makes it possible to add performance measurements even to HttpUnit or DbUnit tests.

In our examples we will use the following unit test

public class SimpleUnitTest extends TestCase {
...
  public void test1() {
      ... //some operations
  }
...

 

A TimedTest can be used to provide a time limit for your running tests. Tests will fail if they take longer than the given time limit to execute. This is useful if you know you don't want to exceed a certain time constraint. Here we are running all tests within the SimpleUnitTest class as a suite with a time limit of 1 second.

public class SimplePerfTest {
  public static void main(String[] args) {
    int timeLimit = 1000;
    Test timedTest = new TimedTest(new TestSuite(SimpleUnitTest.class), timeLimit);
    junit.textui.TestRunner.run(timedTest);
  }
}

 

You can use LoadTest to create a load test with a specified number of threads(JVM will constrain the upper thread limit) running your test case either simultaneously or at randomly dispersed start times. We will do the simplest example of running a test suite with 100 threads concurrently.

public class SimplePerfTest {
  public static void main(String[] args) {
    int load = 100;
    Test loadTest = new LoadTest(new TestSuite(SimpleUnitTest.class), load);
    junit.textui.TestRunner.run(loadTest);
  }
}

 

In a more complex example of using LoadTest, we can simulate 100 threads running our test code, with a random delay before the addition of each concurrent thread. In addition to the RandomTimer class to specify a delay, there is also the ConstantTimer to specify a constant delay between thread addition.

public class SimplePerfTest {
  public static void main(String[] args) {
    int threads = 100;
    int delay = 10;
    int randomVariation = 20;
    Timer randomTimer = new RandomTimer(delay, randomVariation);
    Test loadTest = new LoadTest(new TestSuite(SimpleUnitTest.class), threads, randomTimer);
    junit.textui.TestRunner.run(loadTest);
  }
}

 

Lastly, we can combine both TimedTest, LoadTest, and RepeatedTest (part of JUnit) to string together a rather sophisticated concurrent performance test. Here's a test that concurrently runs your unit test in 100 threads, each repeating 10 times before ending, with a time limit of 2 seconds for the test to complete. Threads are being added after every 10ms by using the ConstantTimer.

public class SimplePerfTest {
  public static void main(String[] args) {
    int timeLimit = 2000;
    int threads = 100;
    int delay = 10;
    int repetitions = 10;
    Timer randomTimer = new ConstantTimer(delay);
    Test timedTest = new TimedTest(new TestSuite(SimpleUnitTest.class), timeLimit);
    Test repeatedTest = new RepeatedTest(timedTest, repetitions);
    Test loadTest = new LoadTest(repeatedTest, threads, randomTimer);
    junit.textui.TestRunner.run(loadTest);
  }
}

 

JUnitPerf tests are great for quickly spotting performance problems early in the development cycle, requiring very little extra work if your regular coding practice already includes JUnit, and extensions of JUnit such as HttpUnit and DbUnit. With multithreaded LoadTests, it can also be helpful in uncovering concurrency issues normally difficult to discover with code inspection or other forms of testing. There is unquestionable value in performing performance testing on a fully functional and integrated system, but you can cut down both cost and risk to your project by validating the performance of your designs early, especially architectural decisions that may be hard to change later on (though being Agilists, we always try to make change easy. But that's a topic for another day).

 

 

Resources:

http://clarkware.com/software/JUnitPerf.html

Home of JUnitPerf

 

http://junit.org

Home of JUnit

 

http://www.dbunit.org/

Home of DbUnit

 

http://httpunit.sourceforge.net/

Home of HttpUnit

 

http://www.ibm.com/developerworks/java/library/j-cq11296.html

Performance testing with JUnitPerf

0 Comments Permalink

In order to maintain persistent shopping carts and wish lists, EP stores session data as customers shop.  Without maintenance, the database tables will grow over time and may lead to performance problems when customers re-visit the site as the system searches through large tables.  This post explains some methods to purge these tables.

 

Of course, we strongly recommend testing these commands in a test environment before touching your production site.  Also, note that some of these commands may take a long time to execute if the tables are very large.

 

A session is created whenever a customer shops. If the customer adds items to a cart or wishlist, the session creates a cart with the associated item(s).  If the customer is identified by cookie or sign-in, the session will reference the customer ID; if not, the session customer ID will be null because the customer is not known.  The database works as follows:

 

Items are linked to carts by a SHOPPING_CART_UID (items contain a reference to the shopping cart primary key)

Carts are linked to sessions by a GUID (reference to session primary key)

Sessions are linked to customers by a CUSTOMER_ID (reference to customer primary key) if the customer is identified; otherwise CUSTOMER_ID is null

 

Because every visit creates a session, the TCUSTOMERSESSION table is the largest and may have sessions with or without known customer ID and with or without a cart.

 

There are a couple ways to purge these tables.  The easiest is simply to delete all entries in the TCUSTOMERSESSION table that have no associated shopping cart or items (i.e. customer browsed but did not add anything to a cart or wishlist).  In SQL, this means deleting all TCUSTOMERSESSION records with a GUID that is not found in the TSHOPPINGCART table.  You should add a time parameter to delete only sessions older than x days - that way you don't delete active or relatively recent carts.  This should get rid of the majority of entries. Look at the data or pull a count before deleting to see how it will affect your store.  The basic SQL looks like this:

 

delete from DB.TCUSTOMERSESSION where GUID not in (select GUID from DB.TSHOPPINGCART);

 

Another more aggressive way is to delete all TCUSTOMERSESSION records where the CUSTOMER_UID is NULL.  This deletes all carts created for unidentified shoppers (whether or not they are empty), but risks orphan carts that have to be cleaned up.  The SQL is:

 

delete from DB.TCUSTOMERSESSION where CUSTOMER_UID is null;

 

This clears out all the sessions for unidentified shoppers, but will likely leave orphan carts in TSHOPPINGCART.  The way to delete these is to first delete all items from all orphaned carts BEFORE deleting the carts, and then to delete all carts whose GUID is not in the TCUSTOMERSESSION table.  The basic SQL would look like below; as you can see, the end of the first command is the same as the second command because first we?re searching for ?sessionless? carts to delete their items and then we?re deleting the sessionless carts themselves.

 

delete from DB.TCARTITEM where SHOPPING_CART_UID in (select UIDPK from DB.TSHOPPINGCART where GUID not in (select GUID from DB.TCUSTOMERSESSION));

 

and then

 

delete from DB.TSHOPPINGCART where GUID not in (select GUID from DB.TCUSTOMERSESSION);

 

Note that these two methods delete different entities: one deletes sessions that never had carts, the other deletes sessions with unidentified shoppers (that may or may not have carts) and then deletes the orphaned carts.

 

Finally, you can delete the (relatively few) carts that had no items in them:

 

DELETE FROM DB.TSHOPPINGCART WHERE UIDPK NOT IN (SELECT SHOPPING_CART_UID FROM DB.TCARTITEM);

 

These can be done on a running store, but remember to add a time parameter so you don't delete current carts that customers are planning to buy.  And remember to test this in your test environment first!

1 Comments 0 References Permalink