Targeted Selling is Elastic Path's personalization engine. It collects data about the shopper from various sources and exposes it for rule-based conditional evaluation. In Elastic Path, this is done via Business User rule configurations. The underlying technology put in place to support Targeted Selling is called the Tagging Framework, enabling the system to perform tag-based customer segmentation.

The following diagram shows how the building blocks fit together and form the basis of the various areas of the Storefront that is exposed to a customer for interaction. As you can see, price lists and dynamic content are both driven by these two frameworks.

Be sure to check out the Tagging Framework and Targeted Selling documentation on the Elastic Path documentation site for details. For our purposes, today's blog entry will consist of an example of how we can customize and extend these frameworks to provide rule-driven personalization for a straightforward use case.

Aa a Sales Engineer, I frequently get asked to personalize our storefront presentation to demonstrate some scenarios that are specific to a prospective client. Here are two recent examples where I needed to set up the system to be configurable and present specific dynamic content for two different customer personas:

• Education Pricing: the system should be able to provide education-level pricing that differs according to which school a customer belongs to
• Membership Levels: the system should be able to provide personalized merchandising and pricing to customers at Bronze, Silver and Gold tiers, as well as Staff or Affiliate pricing
  

Now, I'm quite lazy and would be quite happy to spend my day browsing The Daily WTF. I definitely do not want to start writing new code every time the business/marketing admins want to start segmenting customers via new profile attributes. Therefore, I want this customization to the system to be easily configurable. In this post, we'll look at how I did it.

The steps we'll be running through include:

• Setting up the new customer profile and tagging data to support the customer segmentation
• Setting up segment specific pricing and dynamic content rules
• Extending the system to add a configurable Tagger in the storefront to perform the segmentation
• Testing the new configurable Tagger
• Tying everything together with Spring

Setting Up the Underlying Tag Data

Adding Customer Profile Attributes

I've elected to add two new String-based customer profile attributes via SQL rather than the Commerce Manager client. In this case, we're looking at two custom attributes that are automatically available for editing in the Commerce Manager's Customer editor, as seen below. Optionally, we could have made these attributes System attributes and added custom drop-down boxes with pre-defined values for valid membership levels and schools respectively for the Customer editor. I'll leave that as an optional exercise.

INSERT INTO TATTRIBUTE (UIDPK,ATTRIBUTE_KEY,LOCALE_DEPENDANT,ATTRIBUTE_TYPE,NAME,REQUIRED,VALUE_LOOKUP_ENABLED,ATTRIBUTE_USAGE,SYSTEM,ATTR_GLOBAL)
    VALUES (1000,'CP_MEMBERSHIP',0,1,'Membership Level',0,0,4,0,0);

INSERT INTO TATTRIBUTE (UIDPK,ATTRIBUTE_KEY,LOCALE_DEPENDANT,ATTRIBUTE_TYPE,NAME,REQUIRED,VALUE_LOOKUP_ENABLED,ATTRIBUTE_USAGE,SYSTEM,ATTR_GLOBAL)
    VALUES (1001,'CP_SCHOOL',0,1,'School',0,0,4,0,0);

Adding Custom Tag Definitions and Values

Next we're going to make the Tagging framework aware of new tags for Membership and Schools. We're also restricting the possible allowed values for each respective tag.

INSERT INTO TTAGVALUETYPE(uidpk, guid, java_type) values(20,'school','java.lang.String');

INSERT INTO TTAGVALUETYPE(uidpk, guid, java_type) values(21,'membership','java.lang.String');

INSERT INTO TTAGVALUETYPEOPERATOR(tagvaluetype_guid, tagoperator_guid)
    values('school', 'equalTo'), ('school', 'notEqualTo');

INSERT INTO TTAGVALUETYPEOPERATOR(tagvaluetype_guid, tagoperator_guid)
    values('membership', 'equalTo'), ('membership', 'notEqualTo');

INSERT INTO TTAGDEFINITION(uidpk, guid, name, description, tagvaluetype_guid, taggroup_uid)
    values(30, 'school', 'school', 'school', 'school', 3);

INSERT INTO TTAGDEFINITION(uidpk, guid, name, description, tagvaluetype_guid, taggroup_uid)
    values(31, 'membership', 'membership', 'membership', 'membership', 3);

insert into TTAGALLOWEDVALUE(uidpk, value, tagvaluetype_guid, description, ordering)
     values(42, 'gold', 'membership', 'Gold', 1),
     (43, 'staff', 'membership', 'Staff', 2),
     (44, 'affiliate', 'membership', 'Affiliates', 3);

insert into TTAGALLOWEDVALUE(uidpk, value, tagvaluetype_guid, description, ordering)
     values(45, 'sfu', 'school', 'Simon Fraser University', 1),
     (46, 'ubc', 'school', 'University of British Columbia', 2),
     (47, 'ua', 'school', 'University of Alberta', 3);

Adding Localized Content

For completeness, we're adding localized content so that our Pricelist Assignment and Dynamic Content Delivery rule editors will display our new tags appropriately for administrators.

INSERT INTO TLOCALIZEDPROPERTIES (UIDPK,LOCALIZED_PROPERTY_KEY,VALUE,TYPE,OBJECT_UID)
    values (200000,'tagDefinitionDisplayName_en','is from a school','TagDefinition',30);

INSERT INTO TLOCALIZEDPROPERTIES (UIDPK,LOCALIZED_PROPERTY_KEY,VALUE,TYPE,OBJECT_UID)
    values (200001,'tagDefinitionDisplayName_en','has a membership','TagDefinition',31);

INSERT INTO TLOCALIZEDPROPERTIES (UIDPK,LOCALIZED_PROPERTY_KEY,VALUE,TYPE,OBJECT_UID)
    values (200002,'tagDefinitionDisplayName_fr','is from a school','TagDefinition',30);

INSERT INTO TLOCALIZEDPROPERTIES (UIDPK,LOCALIZED_PROPERTY_KEY,VALUE,TYPE,OBJECT_UID)
    values (200003,'tagDefinitionDisplayName_fr','has a membership','TagDefinition',31);

Adding Tags to the Tag Dictionaries

The system groups tags into tag dictionaries, such as those for pricing and shopper segmentation. We're adding our new tags into these dictionaries so that they show up in the UI for configuration.

INSERT INTO TTAGDICTIONARYTAGDEFINITION(tagdictionary_guid, tagdefinition_guid)
    values('SHOPPER', 'school');

INSERT INTO TTAGDICTIONARYTAGDEFINITION(tagdictionary_guid, tagdefinition_guid)
    values('PLA_SHOPPER', 'school');

INSERT INTO TTAGDICTIONARYTAGDEFINITION(tagdictionary_guid, tagdefinition_guid)
    values('SHOPPER', 'membership');

INSERT INTO TTAGDICTIONARYTAGDEFINITION(tagdictionary_guid, tagdefinition_guid)
    values('PLA_SHOPPER', 'membership');

Configuring the School Pricing

Once this data is all loaded, we can now load up our Price List Assignment editor or Dynamic Content Delivery editor and be able to configure rules based on the new Membership or School tags, such as below.

As such, for a demo I've configured school level pricing for a specific camera to provide a price that is discounted to $150.00 from $199.00. However, this won't show up right away on the storefront because we need to have the School memberships added into a customer's tagset when they are interacting with the storefront.

Creating our own Tagger

To enable the storefront to be aware of the new tags, we're extending the existing CustomerProfileTagger. We're making the custom tagger extensible to automatically add in any configured customer profile attributes into the tagset. Note that the CustomerProfileTagger implements two different listeners for login and session creation events, and thus is executed when these events are fired on the storefront.

package com.elasticpath.sfweb.listeners;
 
import java.util.Iterator;
import java.util.Map;
 
import javax.servlet.http.HttpServletRequest;
 
import org.apache.log4j.Logger;
 
import com.elasticpath.domain.customer.Customer;
import com.elasticpath.domain.customer.CustomerSession;
import com.elasticpath.tags.Tag;
import com.elasticpath.tags.TagSet;
 
/**
 * Applies configured list of tag values into tagset.
 * 
 * @author drewz
 *
 */
public class ConfigurableCustomerProfilerTagger extends CustomerProfileTagger {
 
     private static final Logger LOG = Logger.getLogger(ConfigurableCustomerProfilerTagger.class);
     
     private Map<String, String> profileAttributeTags;
 
     /**
      * Apply configured string-based tags into session's tagset.
      * 
      * @param session instance of CustomerSession
      * @param request the originating HttpServletRequest
      */
     public void execute(final CustomerSession session, final HttpServletRequest request) {
          super.execute(session, request);
          
          if (profileAttributeTags == null || profileAttributeTags.isEmpty()) {
               return;
          }
          
          TagSet tagCloud = session.getCustomerTagSet();
          Customer customer = session.getCustomer();
          
          Iterator<String> iter = profileAttributeTags.keySet().iterator();
          while (iter.hasNext()) {
               String attributeKey = iter.next();
               String tagKey = profileAttributeTags.get(attributeKey);
               String attrValue = customer.getCustomerProfile().getStringProfileValue(attributeKey);
               
               if (attrValue == null) {
                    LOG.debug("Customer attribute " + attributeKey + " not available. Tag value not added to tag cloud.");
               } else {
                    LOG.debug("Adding customer tag " + tagKey + " to tag cloud: " + attrValue);
                    tagCloud.addTag(tagKey, new Tag(attrValue));
               }
          }
     }
     
     public void setProfileAttributeTags(Map<String, String> profileAttributeTags) {
          this.profileAttributeTags = profileAttributeTags;
     }
}

Testing the Tagger

No coding is complete without valid test scenarios! In this case, we're testing that any configured profile attributes are added to the tagset correctly, and any missing/empty profile attributes are not added.

/**
 * 
 */
package com.elasticpath.sfweb.listeners;
 
import static org.junit.Assert.assertEquals;
import static org.junit.Assert.assertNotNull;
 
import java.util.Date;
import java.util.HashMap;
import java.util.Map;
 
import org.jmock.Expectations;
import org.jmock.Mockery;
import org.jmock.integration.junit4.JMock;
import org.jmock.integration.junit4.JUnit4Mockery;
import org.junit.Before;
import org.junit.Test;
import org.junit.runner.RunWith;
import org.springframework.mock.web.MockHttpServletRequest;
 
import com.elasticpath.domain.customer.Customer;
import com.elasticpath.domain.customer.CustomerProfile;
import com.elasticpath.domain.customer.CustomerSession;
import com.elasticpath.tags.TagSet;
 
/**
 * Tests for CustomerProfileTagger.
 *
 */
@RunWith(JMock.class)
public class ConfigurableCustomerProfileTaggerTest {
 
     private static final String MEMBERSHIP_ATTR_KEY = "MEMBERSHIP";
     private static final String SCHOOL_ATTR_KEY = "SCHOOL";
     private static final String MEMBERSHIP_TAG_KEY = "membership";
     private static final String SCHOOL_TAG_KEY = "school";
 
     
     private final Mockery context = new JUnit4Mockery();
          
     private TagSet tagSet;
     private MockHttpServletRequest request;
     private CustomerSession session;
     private Customer customer;
     private CustomerProfile customerProfile;
     private ConfigurableCustomerProfilerTagger listener;
     
     /**
      * Setting up instances.
      */
     @Before
     public void setUp() {
          tagSet = new TagSet();
          request = new MockHttpServletRequest();
          session = context.mock(CustomerSession.class);
          customer = context.mock(Customer.class);
          customerProfile = context.mock(CustomerProfile.class);
          listener = new ConfigurableCustomerProfilerTagger();
     }
     
     /**
      * Test that configured list of profile attributes are added to the tagset. 
      * Testing two tags/attributes for membership and school.
      */
     @Test
     public void testAddExistingProfileAttributeToTagSet() {
          final String membershipValue = "GOLD";
          final String schoolValue = "SFU";
          
          context.checking(new Expectations() { {
               allowing(session).getCustomerTagSet(); will(returnValue(tagSet));
               allowing(session).getCustomer(); will(returnValue(customer));
               allowing(customer).getCustomerProfile(); will(returnValue(customerProfile));
               allowing(customer).getDateOfBirth(); will(returnValue(new Date()));
               allowing(customer).getGender(); will(returnValue('M'));
               allowing(customerProfile).getStringProfileValue(MEMBERSHIP_ATTR_KEY); will(returnValue(membershipValue));
               allowing(customerProfile).getStringProfileValue(SCHOOL_ATTR_KEY); will(returnValue(schoolValue));
          } });
          
          
          Map<String, String> profileAttributeTags = new HashMap<String, String>(2);
          profileAttributeTags.put(MEMBERSHIP_ATTR_KEY, MEMBERSHIP_TAG_KEY);
          profileAttributeTags.put(SCHOOL_ATTR_KEY, SCHOOL_TAG_KEY);
          listener.setProfileAttributeTags(profileAttributeTags);
          listener.execute(session, request);
          
          assertNotNull("Membership tag was null", tagSet.getTagValue(MEMBERSHIP_TAG_KEY));
          assertEquals("Failed to get the correct membership value from the tagset", 
                    membershipValue, tagSet.getTagValue(MEMBERSHIP_TAG_KEY).getValue());
          assertNotNull("School tag was null", tagSet.getTagValue(SCHOOL_TAG_KEY));
          assertEquals("Failed to get the correct membership value from the tagset", 
                    schoolValue, tagSet.getTagValue(SCHOOL_TAG_KEY).getValue());
     }
     
     /**
      * Test that if a customer profile attribute is missing for a customer, no tag value is added to the tagset.
      */
     @Test
     public void testMissingProfileAttributeToTagSet() {
 
          context.checking(new Expectations() { {
               allowing(session).getCustomerTagSet(); will(returnValue(tagSet));
               allowing(session).getCustomer(); will(returnValue(customer));
               allowing(customer).getCustomerProfile(); will(returnValue(customerProfile));
               allowing(customer).getDateOfBirth(); will(returnValue(new Date()));
               allowing(customer).getGender(); will(returnValue('M'));
               allowing(customerProfile).getStringProfileValue(MEMBERSHIP_ATTR_KEY); will(returnValue(null));
               allowing(customerProfile).getStringProfileValue(SCHOOL_ATTR_KEY); will(returnValue(null));
          } });
          
          
          Map<String, String> profileAttributeTags = new HashMap<String, String>(2);
          profileAttributeTags.put(MEMBERSHIP_ATTR_KEY, MEMBERSHIP_TAG_KEY);
          profileAttributeTags.put(SCHOOL_ATTR_KEY, SCHOOL_TAG_KEY);
          listener.setProfileAttributeTags(profileAttributeTags);
          listener.execute(session, request);
          
          assertEquals("Membership tag was not null", null, tagSet.getTagValue(MEMBERSHIP_TAG_KEY));
          assertEquals("School tag was not null", null, tagSet.getTagValue(SCHOOL_TAG_KEY));
     }
     
     
}

Tying it all Together

With our new tagger tested and verified, we now configure it into the system via the Storefront's serviceSF.xml spring configuration by replacing the default customerProfileTagger with our new custom tagger. Note that we're injecting a map of profile attributes that will be added to the tagset.

     <bean id="customerProfileTagger" class="com.elasticpath.sfweb.listeners.ConfigurableCustomerProfilerTagger" >
          <property name="profileAttributeTags">
               <map>
                    <entry key="CP_MEMBERSHIP" value="membership"/>
                    <entry key="CP_SCHOOL" value="school"/>
               </map>
          </property>
     </bean>

     Similarly, in the Commerce Manager application we need the UI to be able to handle the new tags, thus in serviceCM.xml we add in our value providers for the Tag Values.

     <bean id="selectableTagValueServiceLocator" class="com.elasticpath.tags.service.impl.SelectableTagValueServiceLocatorImpl">
          <property name="valueProviders">
               <map>
                              <!-- SNIPPED ... -->          
                    <entry><key><value>school</value></key>
                            <bean class="com.elasticpath.tags.service.impl.InternalSelectableStringTagValueProviderImpl"></bean>
                    </entry>
                    <entry><key><value>membership</value></key>
                            <bean class="com.elasticpath.tags.service.impl.InternalSelectableStringTagValueProviderImpl"></bean>
                    </entry>          
               </map>
          </property>
     </bean>

Tie this together and run the storefront with a test customer with a school membership, and suddenly we have the school level pricing we configured earlier. Custom customer segmentation is easy-peasy! Next time that marketer asks for new segmentation rules, we just go back to our XML file and then take a nice a KitKat break.

Obviously, there's plenty of room for improvement here. Ideally, we make the tags and UI select membership and schools from pre-defined lists. Also, I'm not too fond of having to edit XML files and redeploying the application to handle changes, so we could move the profile attribute configuration into Elastic Path's Settings Framework so that they can be modified at runtime without having to restart. But this is a good starting point, and hopefully that gives you an idea of how easy it is to extend the Tagging Framework to meet some common pricing and personalization scenarios.

Segment away!

Alternative title: How to Make Your Life Easier and Not Go Bald from Pulling Your Hair out Trying to Keep Current on a Constantly Evolving and Shifting Codebase

This is probably the biggest and most avoided question from all of our successfully launched Professional Services projects. How can we upgrade to the greatest and latest on our heavily customized EP project? Let's be clear that performing a point release upgrade, or even consuming the latest sprint of code from our extremely productive Product Development group isn't trivial. A full version upgrade is even more daunting and usually heavily balanced against the cost of re-implementing from scratch.

First off, hopefully you are following our advice to try to avoid customizing EP code as far as possible, as noted in our documentation and tutorials. Following this methodology will make your life a lot easier, as upgrading core EP classes that have only minor changes will be a lot easier.

With that in mind, the best method through tried and tested experience, from brute force patching and merging, to re-implementing large customizations, is to take advantage of the merge tools from your chosen source control software (you are using source control I hope! If not, this book will be your best friend). Why do all the manual merge work when X does it so well already (X being subversion, perforce, cvs, vss, git, the next big flavor of the month). The key to leveraging your source control software to facilitate an upgrade is the concept of a vendor branch.

Working with a vendor branch is straight forward. This is applicable to any 3rd-party library you are using and modifying, not just EP. Take a quick gander at these various interweb sites for a better explanation than i can reasonably provide. For our purposes, a vendor branch for EP is an untainted, pristine copy of EP source. Don't just take our beloved zip of source code and stash it in your network drive! Check it in and show it off! You can branch your customized codeline off the vendor branch and start working away with the enthusiasm of a frenzied squirrel in the spring. The next time a new version of EP comes along, simply drop it directly on top of the current vendor branch of EP in your repository. With this new EP revision, you can perform your normal merge routine of just the diff between these EP revisions and merge the delta of changes into your customized eCommerce solution. And, if you have been following the procedures to avoid modifying EP source, conflicts should be kept to a minimum, files will just resolve themselves, and you can go out for a quick pint. Sure sounds easy huh? But how about in real life?

Well, let's see how we've taken this and made our lives easier at a client site in Perforce. I'm not the biggest Perforce fan, but I sure appreciate it's advanced branching and visual merging capabilities. We've set this up as follows:

• EP_IMPORT, our EP vendor branch containing 100% pure EP 6.1 eCommerce goodness
• Stable - our main development branch, full of customizations of the greatest and latest variety

Now, presume we're taking in change after massive change from our Product Development group. They've given us EP 6.1.1! Hooray! But how do we get this into our source control? Looks easy enough. Drop it on top of EP_IMPORT and check it in? Whoa, hold on there, code cowboy. This baby probably won't compile just with that.

The main things to remember is to collect three things with each new drop of code:

1. All files that have been updated in 6.1.1
2. All files that have been deleted in 6.1.1
3. All files that have been added in 6.1.1

Once you have all three, you can mark the relevant files for delete, add and update. I've taken a short cut to calculate the files that fall into the above three buckets in this case. I have access to our internal subversion repository at EP, so I'm just going to take a log of files taken in by 'svn update' when updating from the 6.1 tag and the new 6.1.1 tag. You can essentially calculate the same bucket of files using the diff command. But in my case I'm just taking a shortcut and doing:

svn update > merge.txt

This logs out for us a lot of lines of this variety:

U    com.elasticpath.core/WEB-INF/src/main/java/com/elasticpath/service/rules/PromotionRuleDelegate.java
D    com.elasticpath.core/WEB-INF/src/main/java/com/elasticpath/service/misc/impl/WorkAroundOpenJPAFetchPlanHelperImpl.java
A    com.elasticpath.core/WEB-INF/src/main/java/com/elasticpath/service/misc/impl/OpenJPAEventListeningFetchPlanHelperImpl.java

With some help from sed (for the non-Unixers, feel free to grab and install cygwin, or your favorite port of the beloved Unix command line tools), I'm going to parse the log and split it into three files:

cat merge.txt | sed \-e '/D /d' | sed \-e '/U /d' | sed \-e 's/A[ ]*//' > adds.txt
cat merge.txt | sed \-e '/A /d' | sed \-e '/U /d' | sed \-e 's/D[ ]*//' > deletes.txt
cat merge.txt | sed \-e '/A /d' | sed \-e '/D /d' | sed \-e 's/[ ]*[U]*[ ]*//' > updates.txt

Now, Perforce is a bit finicky in that I need to mark which files I'm going to edit to remove the read-only permissions of relevant files. Sure, I can mark every file for editing, but let's play it safe:

pwd
/perforce/depot/branches/EP_IMPORT/
cat updates.txt | xargs p4 edit -f

Next, time to mark the deleted files for removal:

cat deletes.txt | xargs p4 delete -f

Finally, it's time to unzip on top all the files from the new EP 6.1.1 source zip and mark our new files for add:

tar -xvzf EP6.1.1_Export.tgz .
cat adds.txt | xargs p4 add -f

That does it! Now check that all in! Now we have EP 6.1.1 in our EP_IMPORT branch. Time to use the impressive P4V tool from Perforce to integrate from the EP_IMPORT branch into the Stable branch. Since we've avoided modify EP source as much as possible, the majority of the changes should auto-resolve. Clean up some of the conflicts and tweak the customizations as necessary, run the normal routine of compile, checkstyle, pmd, unit tests, integration tests and your project should now be ready to go. Okay, that's a gross over-simplication of the conflict resolution and regression testing process that goes hand in hand with a version upgrade, but hopefully you see how a vendor branch does wonders in facilitating an easier upgrade process with custom code. No more awkward patch files and brute force.

Assuming you don't have access to our subversion to get a hold of the list of changes, you can use diff to log file differences and files missing from the 6.1 and 6.1.1 codebases.

diff -r /perforce/depot/branches/EP_IMPORT /tmp/EP611/ > diffs.txt

This should give you a nice log that is easily parsed into the same add/delete/update buckets. So now you're freshly armed to do the upgrade that has been haunting you late at night. Vendor branches and merge tools are your new best friends! Happy upgrading and merging.

Drew