Technical Blog

12 Posts tagged with the 6.1 tag

This post walks through adding filtering to the System Configuration page in the CM Client helping you track down a setting as quickly as possible.

 

We added the Settings Framework in Elastic Path Commerce 6.1 and it's proved incredibly useful in centralizing configuration and easing customization versus the previous XML based approach.  The number of settings out-of-the-box continues to grow plus any additional ones that customers are free to add.  So this tip will show you how to reign in that growing number of settings.

 

Here's a screenshot of what we'll be producing, if you've spent any time in the System Configuration (like our trusty QA guys) this will be a real time saver.

 

settings-filter.png

 

Changing the layout

I had to refresh my SWT widget layout knowledge a little and found this great page: http://www.eclipse.org/articles/article.php?file=Article-Understanding-Layouts/index.html.  With that excellent refresher I decided I would need a three column GridLayout: a column each for the Edit button, filter label and the filter text widget.  The table would then span all three columns.

 

settings-filter-layout.png

First, simply change the number of columns on the SettingDefinitionComposite:

 

private void setupLayout() {
     final int columns = 3;
     this.setLayout(new GridLayout(columns, false));
}

 

Then add in a horizontal span to the table's LayoutData - this makes it span the three columns we just created.

final int horizontalSpan = 3;
table.setLayoutData(new GridData(GridData.FILL, GridData.FILL, true, true, horizontalSpan, 1));

Note: the final variables are to keep Checkstyle quiet about magic numbers.

 

Adding the new widgets

The code below adds the label and the filter Text widget.  Take a close look at the style we are using to create the Text widget with: SWT.SEARCH | SWT.CANCEL.  The SWT.SEARCH style gives us the rounded edges (at least on my Mac) which makes it look like a regular search/filter widget.  The SWT.CANCEL style adds the small cross to the widget.  When clicked it removes the currently entered text.

Label label = formToolkit.createLabel(this, AdminConfigurationMessages.filterLabel + ":"); //$NON-NLS-1$
GridData gridData = new GridData(SWT.END, SWT.CENTER, false, false);
label.setLayoutData(gridData);
Text settingNameFilter = formToolkit.createText(this, "", SWT.SEARCH | SWT.CANCEL); //$NON-NLS-1$
gridData = new GridData(SWT.FILL, SWT.CENTER, false, false);
final int verticalIndent = 10;
gridData.verticalIndent = verticalIndent;
settingNameFilter.setLayoutData(gridData);

 

We want this new addition to be localizable so we need to add AdminConfigurationMessages.filterLabel

public static String filterLabel;

 

And then we provide the English version of that in AdminConfigurationPluginResources.properties

 

filterLabel=Filter

 

Filtering

Now we'll take a look at how we will actually filter down the table's contents.  The code's pretty simple:

/**
 * Filter setting definitions against a specified string.
 */
private class SettingPathFilter extends ViewerFilter {
     private final String filterText;
     public SettingPathFilter(final String filterText) {
          this.filterText = filterText;
     }
     @Override
     public boolean select(final Viewer viewer, final Object parent, final Object element) {
          SettingDefinition definition = (SettingDefinition) element;
          return StringUtils.containsIgnoreCase(definition.getPath(), filterText);
     }
}

We've simply extended the JFace ViewerFilter class and implemented the select method with a case-insensitive check against the setting definition's path.  In the next step we'll create an instance of this class and pass it to the TableViewer that holds the setting definitions.

 

The following page helped me a bit with the ViewerFilter: http://www.java2s.com/Code/Java/SWT-JFace-Eclipse/DemonstratesListViewer.htm

 

Hooking it together

The bits are all in place, let's tie them together and get the filtering going when a user types in the text box.  We simply add a ModifyListener to the filter Text widget and when we receive a ModifyEvent we grab the user-entered string, create a new SettingPathFilter and set that on the tableViewer.  That triggers the tableViewer to filter its contents.  Note: I chose to use setFilters, rather than addFilter/removeFilter, to keep the code simple: I don't have to keep track of the filter to remove it afterwards I simply call setFilters again.

 

settingNameFilter.addModifyListener(new ModifyListener() {
     public void modifyText(final ModifyEvent event) {
          final Text source = (Text) event.getSource();
          String filterText = source.getText();
          if (StringUtils.isBlank(filterText)) {
               tableViewer.setFilters(new ViewerFilter [0]);
          } else {
               tableViewer.setFilters(new ViewerFilter [] {new SettingPathFilter(filterText)});
               }
     }
});

 

 

Conclusion

So there you have it, all done, with a few small changes we can find settings without having to visually scan the table.  Where else might this filtering function be useful?  Anyone done any similar customizations they would like to share?

 

The code

The code changes and the attached patch are against the upcoming 6.2 release, but I'm pretty sure they will apply to any 6.1+ version.  Leave a comment if you have any problems applying this and I'll do my best to help you out.

References

http://www.eclipse.org/swt/snippets/ - if you've not taken a look at the SWT snippets then you're missing out.  There's a ton of useful examples of using SWT.

http://wiki.eclipse.org/index.php/JFaceSnippets - just like the SWT snippets, this page contains great snippets about using the JFace ui toolkit.

2 Comments Permalink

We often get asked whether Elastic Path can be integrated with a CMS. We always answer, "yes, of course!" but I suspect at least a few of us wonder how much effort is really involved. I know I did. Well, now I can say that the effort is minimal (no coding required!), at least if you're using Alfresco. Alfresco is a production-ready, open source content management system, and in this post, we'll see just how easy it is to install Alfresco, import your assets, and set up Elastic Path to pull assets from Alfresco.

 

First, you'll need to get Alfresco on your machine. Alfresco comes in two flavors:

  • Enterprise Edition, which includes full commercial support
  • Community Edition, which is unsupported and intended for developers who just want to play around with it.

 

Download the Community Edition (at the time of writing, the current version of the Community Edition is 3.2). Note that although the Community Edition is unsupported, you can still ask questions in the Alfresco forums. The Alfresco wiki is also an excellent source information.

 

When you run the Alfresco setup, it will ask you to specify the setup type. To keep things simple, choose typical. This installs all the required components including Java, Tomcat, and MySQL. If you already have Tomcat installed, you may need to change the Server and Connector ports to avoid conflicts. You can do this after the installation is complete by editing tomcat\conf\server.xml under your Alfresco installation directory.

 

After it's installed and running, go to the login screen (http://localhost/alfresco) and enter the administrator username and password (admin/admin).

 

After logging in, go to the Company Home area and create a space to store the assets for your Elastic Path deployment. Under Browse Spaces, click Sites. (You can choose Web Projects instead if you prefer.)

alfresco_browse_spaces.png

Click Create->Create Space.

alfresco_create_space.png

Enter a name and title for the space and click Create Space. Your space now appears under Browse Spaces.

alfresco_new_space.png

Now you need to add your Elastic Path assets to your space. The fastest way to do this is to zip your entire assets directory and import the zip file into the space.

 

First click the space name. Then click More Actions->Import.

alfresco_import.png

Browse to the assets zip file, select it, and click OK. After the zip file is imported, the contents are extracted automatically and the directory structure is recreated within the space. It might take a few minutes to complete, so the folder may not appear in Alfresco right away. (You can delete the zip file after the assets folder is created.)

 

Now we need to get the SMB/CIFS path that we can use in Windows Explorer to access the assets. Click the View Details icon under the assets folder.

alfresco_view_details.png

If you're using Internet Explorer, click View in CIFS. (Because it's using a UNC path, it won't work in other browsers.)

alfresco_view_in_cifs.png

In the Community Edition, all the supported repository access protocols are enabled by default, including FTP and SMB/CIFS. Unless you've done something strange, you should see a Windows Explorer window showing the contents of the assets folder in Alfresco. (If you're not using Internet Explorer, copy the CIFS link from your browser and paste it into Windows Explorer.)

alfresco_explorer.png

Take note of the path in the address box. It should start with the Alfresco server name followed by an "a". That "a" is automatically added by Alfresco, but it can be configured by changing the cifs.serverName property in tomcat\webapps\alfresco\WEB-INF\classes\alfresco\subsystems\fileServers\default\file-servers.properties , located under your Alfresco installation directory. For more information, see http://wiki.alfresco.com/wiki/File_Server_Subsystem.

 

The next step is to map that path to a Windows drive letter. (The COMMERCE/SYSTEM/assetLocation setting in Elastic Path must be set to a valid file system path; UNC paths are not supported.) In Windows Explorer, click Tools->Map Network Drive. Select an unused drive letter and enter the path in the folder field. Make sure Reconnect at logon is selected. Then click Connect using a different user name. Enter the Alfresco administrator user name and password. Click Finish.

alfresco_map_net_drive.png

The final step is to log on to the Commerce Manager client and configure the COMMERCE/SYSTEM/assetLocation setting to point to the assets folder on the mapped drive. After you do this, restart your app server. Your storefront will now be pulling its theme files, images, etc. from Alfresco. You'll also need to point the different VFS settings in Elastic Path to use Alfresco's FTP so that the CM client's asset manager is able to retrieve the assets from Alfresco as well.

 

Once your assets are in Alfresco, you'll be able to take advantage of its sophisticated content approval workflow and collaboration features.

11 Comments Permalink
Background

 

Our client's e-commerce store has been powered by Elastic Path for well over a year now. Our team started development of the store on a beta version of Elastic Path 6.0 in January 2008. After 3 months of development, including a merge with the public release of Elastic Path 6.0, we launched the store in March 2008. We continued to add new features and bug fixes. We even took advantage of EP 6.0's multi-store support to add a few more stores in August 2008. At the beginning of 2009, after a year's worth of customizations atop the client's EP 6.0 deployment, we decided that it was time to upgrade to EP 6.1.
timeline.jpg



Why Upgrade?

 

Upgrading a customized version of Elastic Path Commerce is not a trivial process. So you need to ask yourself (and the client) if an upgrade is even necessary. The first step of upgrading Elastic Path is determining whether the benefits of upgrading exceed the costs of upgrading. The cost is time and money.

 

What are the benefits? Here are some of ours:
  • EP 6.1 provides full multistore. In EP 6.0, multiple stores required multiple storefront WAR files: one for each store. EP 6.1 runs multiple stores from a single WAR file. Less WAR files translates to quicker deployments and a smaller resource footprint.  This is especially relevant to us since our client is running six stores.
  • EP 6.1 comes with 300+ bug fixes.
  • Upgrading to EP 6.1 paves the way for upgrades to future versions of Elastic Path. This is crucial, because our client is anticipating features that are planned in future versions of EP (e.g. import/export of promotion data, available in EP 6.1.1).
  • EP 6.1 comes with FIT integration tests.
  • EP 6.1 introduces the import/export utility, used to migrate catalog data between databases.
  • EP 6.1 introduces the settings framework, an improved way to manage configuration settings.
  • EP 6.1 allows search index rebuilding to be triggered from the CM Client.
  • The Store configuration UI has been overhauled in EP 6.1; the Store wizard has been replaced by a Store editor.
  • In EP 6.1, assets (e.g. velocity templates, images, etc.) are stored outside the web application, making the view layer much easier to modify.

 

We presented a list of EP 6.1 benefits to our client, provided an estimate on the amount of time it would take to complete the upgrade, and got the green light to go ahead with it. These were the benefits relevant to us, but EP 6.1 has many other improvements. For example, here are 7 Big Code Changes in EP 6.1.



The Upgrade

 

After we convinced ourselves and our client that upgrading to EP 6.1 was worthwhile, we moved on to the fun part: the upgrade itself. There are several steps that need to be completed during an upgrade and it's easy to become overwhelmed at first, so we broke the upgrade down into more manageable sub-tasks:
  1. Database updates
  2. Merging the code
  3. Moving customized named database queries
  4. Using the new settings framework
  5. Relocating the assets directory
  6. Setting up Maven
Let's cover each sub-task in more detail.

 

1. Database Updates
The database schema did not change significantly between EP 6.0 and EP 6.1. Elastic Path has a script that upgrades the database from EP 6.0.x to EP 6.0.3 and another that upgrades it from EP 6.0.3 to EP 6.1. We ran both of these scripts against our database with no problems. Most of the schema changes consists of new tables and new columns and there are few changes to existing columns. Click here for detailed information about the EP 6.1 database updates.

 

After upgrading the schema with the scripts, a few data updates needed to be made:
  • By default, full credit card numbers are stored in the database for each order; we turned this off (the ability to turn it off is a new EP 6.1 feature). This is accomplished by setting STORE_FULL_CREDIT_CARDS to 0 for each row in TSTORE.
  • Populate the new TSTORESUPPORTEDLOCALE table with store-locale mappings; these mappings mirrored the catalog-locale mappings already in TCATALOGSUPPORTEDLOCALE.
  • Populate the new TSTORESUPPORTEDCURRENCY table with the store-currency mappings; these mappings are similar to the catalog-currency mappings already in TCATALOGSUPPORTEDCURRENCY.

 

2. Merging the Code
The majority of the time spent on the upgrade was in this step: merging code. Generally, the more changes you make to a codebase, the harder it is to upgrade in the future. I say "generally" because there are ways to customize code that minimize future code merge conflicts. For example, this GREP technical blog post shows how to customize code with the decorator design pattern; a strategy that significantly reduces code conflicts.

 

The easiest way to merge large amounts of code is by using your version control software's merge tools combined with vendor branches. We have a great GREP article explaining what vendor branches are and how they can be used for merging code here. Since we use subversion for this particular project, we ended up using svn merge to merge the code.

 

This is the svn merge command we used:
svn merge https://svn.elasticpath.com/perfect_code/pd/ep5/tags/publicrelease6.0/ https://svn.elasticpath.com/perfect_code/pd/ep5/tags/publicrelease6.1/ . --accept postpone
This command takes the differences between OOTB EP 6.0 and OOTB EP 6.1 (whose vendor branches are specified in green and blue respectively), and applies those differences onto our customized codebase (specified in red, this command was run from our project's root level directory). The --accept postpone part tells the utility to postpone resolving any merge conflicts that occur.

 

The following diagram shows the branches referred to in the svn merge command and our desired "Customized EP 6.1" end state:
codemerge.jpg
When svn merge hits a conflicted file, it produces 3 output files. For example, consider a file called MyClass.java that is conflicted; svn merge will produce the files MyClass.java.left, MyClass.java.right, and MyClass.java.working. The highlighted colors correspond to the branches in the svn merge command above. The .left file is the EP6.0 version, the .right file is the EP 6.1 version, and the .working file is our customized codebase version.

 

The utility does a pretty good job at merging the source files. Some stats we gathered after running this command:
  • 3682 files were added
  • 118 files were deleted
  • 953 files were automatically merged successfully
  • 132 files were conflicted and not automatically merged
The high number of files added is largely attributed to the relocated assets directory and the new FIT tests. The 953 files merged successfully demonstrates the advantage of using a utility to automatically merge files. Unfortunately, not everything can be automatically merged; we had to manually merge the 132 conflicted files. These numbers will vary depending on the type and amount of customizations you've made on your codebase; our codebase has a lot of customizations.

 

There are many approaches to manually merging a conflicted file. The .left, .right, and .working files produced by svn merge, described above, are invaluable to this process.

 

This is my strategy:
  1. diff (i.e. compare) the .left file (EP 6.0 version) with the .right file (EP 6.1 version); this shows Elastic Path's changes to the file from version 6.0 to 6.1.
  2. diff the .left file (EP 6.0 version) with the .working file (your customized version); this shows your customizations to the file from 6.0 to its current state.
  3. Based on the comparisons above, who changed the file more? Elastic Path or you?
    • If Elastic Path made more changes to the file, start with the .right file and incorporate your changes (indicated by the diff in step 2) to the file.
    • If you made more changes to the file, start with the .working file and incorporate Elastic Path's changes (indicated by the diff in step 1) to the file.
    • If they have the same amount of changes, it doesn't really matter which version you pick as your starting point; the amount of work will roughly be the same. I lean towards picking the .right file (EP 6.1 version) because aligning your codebase to Elastic Path's will make future upgrades smoother.
  4. After merging the file in step 3, you are done. Move onto the next file!
More numbers. Of the 132 conflicted files:
  • 88 were relatively easy/trivial merges
  • 23 were moderately complicated merges
  • And 21 were very complicated merges
The very complicated merges were the files that were changed significantly by both us (i.e. our customizations) and by Elastic Path (during the development of EP 6.1). These files include the class files for ShoppingCart, OrderService, CheckoutService, and other areas of Elastic Path that are often modified.

 

All in all, the code merge is likely not as painful as most anticipate it to be. Even with 40+ files requiring non-trivial manual merges, our code merge went pretty smoothly.

 

A couple tips:
  • Take advantage of dry-run modes of your merging tool to preview how much work needs to be done without actually performing the merge. For example, adding the --dry-run option to our svn merge command enabled us to see how many merge conflicts we'd have to resolve in advance of the merge itself; this information is very useful during the estimation phase.
svn merge https://svn.example.com/project/tags/publicrelease6.0/ https://svn.example.com/project/tags/publicrelease6.1/ . --accept postpone --dry-run
  • As we manually merged the conflicted files, we took note of the merge details in a list. For each file that required non-trivial merging, we jotted down a couple sentences describing the merge (e.g. which methods were merged, etc.). Taking notes during the code merge requires a little overhead, but the list proved to be very useful when bugs caused by the merge were found later on.

 

3. Moving Customized Named Database Queries

 

In EP 6.0, named JPA database queries were embedded in the class files. For example, this bit of code is in CatalogImpl.java:
@NamedQueries({
    @NamedQuery(name = "CATALOG_IN_USE_BY_PRODUCT_TYPE",
            query = "SELECT pt.uidPk FROM ProductTypeImpl pt"
                    + " LEFT JOIN pt.catalog c WHERE c.uidPk = ?1")
    ...
})

In EP 6.1, the named JPA database queries have been moved from the class files to separate object-relational mapping XML files. The query above has been moved to the catalog-orm.xml file.
The code merge should take care of the OOTB (out-of-the-box) named queries but any customized name queries that we added had to be manually moved from the class file to the appropriate XML file. 


4. Using the New Settings Framework

 

In EP 6.0, configuration settings were stored in various XML files (e.g. commerce-config.xml, default.xml, etc.). EP 6.1 introduces a new settings framework that stores settings in the database and allows them to be managed from the CM Client; the setting values are retrieved wherever they are needed in the code via the SettingsService class. The new settings framework is a big improvement for managing configuration settings, but some work needs to be done to take advantage of it.

 

First, some of the OOTB setting values need to be overridden with your project's values. For instance, our client's stores use Elastic Path's one-page checkout. There is a boolean setting that turns this feature on. By default, this setting is off. In EP 6.0, we turned this setting on by modifying a line in commerce-config.xml: onepage.enable=true. In EP 6.1, this setting value is no longer retrieved from commerce-config.xml (although the value may still be there depending on how it was handled during the code merge), but it is not being used. The setting value is now retrieved from the database via the new settings framework. We located the setting (COMMERCE/STORE/enableOnePageCheckout) and changed its value from false to true with a database update. After we were done modifying the OOTB setting values, we removed all unused setting definitions from the XML files.

 

Second, we had to migrate all of the custom settings we created to the new settings framework. We have approximately 60 custom settings -- much more than typical projects. Migrating custom settings is not absolutely required since they are still parsed from XML files and retrievable using the ElasticPath class, but there are many advantages of using the settings framework.

To migrate a custom setting, you first need to create the setting definition with a database insert statement. For instance, the following insert statement creates our "tradeshow timeout" setting:
INSERT INTO TSETTINGDEFINITION(UIDPK, PATH, DEFAULT_VALUE, VALUE_TYPE, DESCRIPTION, MAX_OVERRIDE_VALUES)
 VALUES(81, "CUSTOM/tradeshowTimeout", "120000", "Integer", "Indicates the tradeshow timeout interval, in seconds.", -1);
Next, find all the places in the code that use this setting. Modify the code to use the SettingsService to retrieve the setting value from the database instead of from the XML files (via ElasticPath). Going back to the example above, we make the following javascript change:
timeoutId = window.setTimeout("resetTradeshowPage()", $ctxStoreConfig.getSetting("CUSTOM/STORE/tradeshowTimeout").getValue());
...and the following java code change:
final String settingValue = getSettingsService().getSettingValue("CUSTOM/STORE/tradeshowTimeout").getValue();
 
During our merge, we went one extra step and implemented environment-specific settings, a feature that is not available out of the box and that required some additional customization. You can read more about environment-specific settings in this technical blog post on GREP.

 

5. Relocating the Assets Directory

 

In EP 6.1, the assets directory was moved outside of the war file. For example, storefront templates were previously located in com.elasticpath.sf/WEB-INF/templates/; they are now located in assets/themes/storecode/default/templates. Note that the storefront templates were not the only ones that were relocated; CM templates, like those used for order confirmation emails, were also moved from the com.elasticpath.sf project to the new assets directory. The relocation decouples the view (i.e. templates, images, etc.) from the model and controller (the application). There are many advantages to this, including the ability to easily hot-swap template changes. However, the relocated assets directory made the code merge more complex.

Depending on your merging tool, the automated code merge may handle the relocated assets directory differently. We used svn merge. Unfortunately, svn merge was not smart enough to realize that the assets were relocated; it assumed that the assets directory was simply removed from its old location and that a new assets directory was added to its new location. As a result, the files were not merged at all. The new assets directory was the OOTB version.

 

Thus, we needed to manually merge all of our template changes. This process is not as bad as it sounds, since Elastic Path did not make any major changes to the templates from EP 6.0 to EP 6.1. We merged most of the templates by first diffing the EP 6.0 and EP 6.1 versions and then adding those differences to our customized versions.
Some of the bigger template changes made between EP 6.0 and EP 6.1 include:
  • Retrieving the store and catalog not from the ElasticPath object anymore but from the new StoreConfig object
  • Using the #emailMessage macro (instead of the #springMessage macro) in the email templates to display localized messages
Also worth mentioning is the new layout of the assets directory. Previously, all templates where put into a single templates directory. In EP 6.1, there is support for multiple themes and store-specific templates. For instance, the view-cart.vm template file may live in /assets/themes/mytheme/store1/templates/velocity/shoppingCart; this means that the store whose storecode is "store1" and whose theme is "mytheme" will use that particular view-cart.vm template. Another store, whose theme setting is set to "anothertheme"and doesn't use store-specific templates would have its view-cart.vm template in /assets/themes/anothertheme/default/templates/velocity/shoppingCart. During the merge, we made sure to set up the assets directory structure in a way that would best work for us.

Note: I mentioned above that template files can now be easily hot-swapped. One thing to keep in mind is that by default, templates are cached when the application starts up. If you hot-swap or modify a template file, you need to force a cache refresh by invalidating the cache. A quick and easy way to do this is to open the
invalidate-cache.ep URL in a web browser.

 


6. Setting up Maven

 

EP 6.1 uses Maven to manage project dependencies. In the past, if a project required a component (e.g. a 3rd party library) that was not used OOTB, we would put that component in the libs directory. With Maven, there's no longer a need for a libs directory, so we were able to remove it altogether during our merge. There was one caveat however: we were using a 3rd party library for GeoIP lookups (Maxmind). We added the dependency to our GeoIP library by adding the following in the storefront's pom.xml file:
<dependency>
    <groupId>maxmind</groupId>
    <artifactId>geoip</artifactId>
    <version>1.2.1</version>
    <scope>compile</scope>
</dependency>
The storefront is now dependant on the Maxmind GeoIP component. However, Maxmind is not available OOTB so we also had to make sure that Maven would be able to download the library files from somewhere.
Elastic Path uses Archiva repositories to store the dependent libraries. Rather than adding the Maxmind files to the internal Elastic Path repositories, we decided to add it to our team's Archiva repository. Archiva has a nice web interface to upload jar files into the repository. We added the following to elasticpath.pom to tell Maven that it needed to look in our team's archiva repository (maven-ps.elasticpath.net) for some of its dependencies:
<repository>
    <id>ps-extras</id>
    <name>Elastic Path PS Extras Repository</name>
    <url>http://maven-ps.elasticpath.net:8080/archiva/repository/extras/</url>
    <releases>
        <enabled>true</enabled>
    </releases>
    <snapshots>
        <enabled>false</enabled>
    </snapshots>
</repository>

To sum up, setting up Maven should require no work if your project does not have any custom dependencies. If your project does have dependencies that are not included with OOTB Elastic Path, they will need to be defined in the project object model (POM) files.

 


Testing & Bug Fixing


Our QA team's first task was to verify that EP 6.1 worked on our test servers. We grabbed the OOTB public release version of EP 6.1, deployed it on our test machines, and ran through a smoke test. Once we were able to confirm that the OOTB EP 6.1 worked on our servers, we focused on our merged codebase.

 

The code merge modified numerous files across the entire project. As a result, a full regression test on all of our customizations was required to verify that the code merge was done correctly. It goes without saying that the full regression test took a considerable amount of time to complete. The faster the code is merged, compiled cleanly, and deployed to test servers, the sooner testing can begin. After merging the code, we pushed a testable build out as soon as we could so that our QA team could begin testing. The first half-dozen builds contained many blocking bugs, but they were still testable. The key is to fix the blocking bugs quickly so that the QA team can be kept busy with the full regression test.

 

The JUnit tests should also be run and fixed as soon as possible, preferably even before the first testable build is deployed; they are a great way to catch bugs at the source code level.

 

The new FIT tests in EP 6.1, however, were a little more difficult to run. They require some setup and many may be broken after the code merge depending on the customizations. We skipped the FIT tests during the code merge and instead chose to run them after the upgrade.

 


Time Estimates

 

There's no definitive answer for how long an upgrade will take. It depends on the amount and type of customizations. For our client's upgrade, the entire process -- including scoping, upgrading, testing, and bug fixing -- took three months. One developer completed the upgrade (i.e. database updates, code merge, etc.), two QA analysts performed the testing, and three developers tackled the bugs found by the testers. 8% of our budget was spend on scoping and estimating, 60% was spend on the actual upgrade tasks and bug fixing, and 32% was spent on testing.

 

These numbers will vary from project to project. For example, the client's custom settings required additional custom work, which took approximately two weeks to design and implement. This task is something we developed specifically for this project only.

 


Conclusion

 

Upgrading Elastic Path is neither trivial or impossible. Elastic Path provides documentation on the various aspects of upgrading to Elastic Path 6.1 (database updates, settings migration, assets relocation, etc.), but the complete story is different for each project. The amount of work depends on the number and type of customizations that have been made. If you have any questions or comments, feel free to post them here.

 

0 Comments Permalink

The intent of this post is to clear up some foggy notions and misconceptions about the Elastic Path 6.1 development environment, as well as provide a few insider tips and tricks. Developers of CM Client modules will probably find the following more helpful than web or core application developers, but the principles can be used by everyone.


If you're doing any development of the CM Client and you make a change to EP Core, and you want that change to be available in the CM Client, your typical build time shouldn't be more than a minute or so, hopefully much less. Let's clarify:

When you're running the CM Client from within Eclipse, the CM Client will not use the CORE jar; it will use the CORE project's class files directly.
This means that you do NOT have to run "ant all" on the command line (I never run it - it takes forever).

Let's look at a typical day of developing some new feature in the CM Client where you might be changing some code in the EP Core project. If (and only if) you have changed any persistent classes in the core (i.e. domain objects), you have to make sure that JPA has "enhanced" these classes before you can use them, right? So you have two choices (maybe more, but these are the two that I see):

  1. run "ant enhance" in the com.elasticpath.core project and refresh the project in eclipse. This ensures that JPA has a chance to modify the class files that eclipse's compiler created and then eclipse doesn't think its knowledge of the class files is out-of-date. This takes a few seconds.
  2. run "ant build" in one of the web projects (e.g. com.elasticpath.cm), which will rebuild the core project, enhance its class files, and refresh the core jar file that the webapp will use if you're running it outside of eclipse (which most of us don't do anyway - we run it with WTP inside eclipse, so once again the project files are used rather than the core jar file). This takes about 15 seconds on my box (unless maven connections are having problems, but that's a separate issue).


So in a typical developer environment for EP 6.1, using WTP to run the webapps, the CORE jar is never actually used. So it doesn't have to be built, and it certainly doesn't have to be copied anywhere for the CM Client to use it.

Now for some caveats:

  1. If you have previously built the CM Client on the command-line (ant all, ant build-product, etc) you will likely see that it has created a jar file for each plugin in each of the plugin directories. You must delete these or you will find that changes to the code for that plugin are not reflected in the running copy (if it finds a jar, it will use that jar instead of your project code).
  2. The com.elasticpath.ql project seems to be referenced as a jar file from the com.elasticpath.cmclient.libs project. You must build this project once on the commandline so that its jar is created; otherwise you'll find that running the CM Client won't work because the Spring initializer is expecting some code that it cannot find. This often catches developers who have just imported all their eclipse projects and they try to run the CM Client. If you're developing on this ql project you likely already know this; if you're not, then you only have to do this once and you can forget about it.
  3. When setting up a new workspace, you *do* have to do an "ant eclipse-setup" once in the com.elasticpath.cmclient directory to make sure that all the CM Client plugins have eclipse project files generated for them, which you can import into eclipse. The list of projects for which project files will be generated lives in the build.xml file in that directory. If you're adding a new plugin, you will need to add it to that list so that its project file is generated properly (with maven information, etc).
  4. Each Web project requires a META-INF/context.xml file for you to be able to run it in tomcat with WTP. This file is generated by the eclipse-setup script if you've got tomcat specified in your env.config file when you run the script, but I believe it's wiped out when you run "ant clean"... so don't run that, or if you do then make sure you run ant eclipse-setup again.



0 Comments 0 References Permalink

Many classes were refactored in Elastic Path 6.1 to improve testability of existing code and to accommodate new features (most notably the ability for a single Storefront WAR file to host multiple Stores). Here are the changes with the highest impact for developers (in no particular order).

1. Refactoring of ElasticPathImpl.java and Creation of the Settings Framework

ElasticPathImpl covers several distinct areas of responsibility: BeanFactory, Configuration, Filtered Navigation, asset lookup... In 6.1, new interfaces were created to move those areas of functionality out of ElasticPathImpl with the intention of rendering the object obsolete in future versions.

  • Most configuration settings are now accessed via the SettingsService, which will be covered in a separate article. Commerce-config.xml files are no longer used to populate a map of settings in ElasticPathImpl, with the exception of the one in the CM web application, which is only used to keep track of the EP version and build number
  • Asset locations are now retrieved from the AssetRepository interface (see heading elsewhere in this article)
  • Search Configuration is now retrieved entirely from the Settings framework rather than from the ElasticPath object
  • Filtered Navigation information is still retrieved from ElasticPathImpl, but only because it implements the new FilteredNavigationConfiguration interface. The actual retrieval implementation now lives in a separate class and references the new Settings framework
  • Bean instances can still be retrieved from ElasticPath because ElasticPathImpl implements the BeanFactory interface, but from this point onward the ElasticPath object is not required to be injected into classes that only need a BeanFactory. Instead, you should inject an implementation of BeanFactory

If any customizations were performed on some Elastic Path 6.0 code base that must be transferred to an Elastic Path 6.1 code base, most careful attention should be paid to any changes in the ElasticPath singleton, especially if any new settings were introduced that are designed to be read in from a commerce-config.xml configuration file. Since most settings were moved to the Settings framework, the code to parse the configuration file has mostly been removed.

New customizations should pay special attention to the Settings framework and use it where appropriate. Also, the BeanFactory should be injected into new classes where required instead of the ElasticPath object, as its use as a bean factory is being phased out.

New persistent classes should have a DAO that is separate from any new services, to allow for more flexible persistence implementations if functionality changes.

2. Testing Framework Enhancements

In testing, the big news is deprecation of ElasticPathTestCase.java and the introduction of JUnit4 and JMock2 testing frameworks.

Prior to 6.1, many test classes extended ElasticPathTestCase.java, which is a JUnit 3.8 extended test class that mocks out much of the application's domain using mock objects created with the JMock 1.x framework. This class is being phased out, and use of it is no longer recommended because it:

  • is large and unwieldy
  • encourages poor code factoring and integration tests using mock objects
  • is complex; modification may affect many test cases in an unforeseen manner

 

Instead, unit test classes should focus on unit-testing rather than integration-testing methods. Use of the JUnit4 and JMock2 framework features should help in that regard. There are many examples of their use in the new code; but for a start you may want to look at GiftCertificateThemeImplTest (for a relatively simple test class) and StoreResourceManagerImplTest (which takes advantage of some advanced features in the new JMock framework).

3. New Store UI Themes and Asset Retrieval

Store-specific Velocity template resolution has been introduced. Most static and storefront theme content (storefront velocity templates, email velocity templates, javascript files, css files, Power Reviews, etc) was moved outside the separate application WAR files into the pre-existing Assets folder on the file system.

 

In 6.0 only some assets (store images and digital goods) lived on the file system under the assets folder, while others (email templates, storefront UI files, and PowerReviews files) lived inside the WAR. All of these files were moved to appropriate locations under the file system's assets folder and the paths to such folders were made available through an AssetRepository object rather than through methods on the ElasticPath object. This allows the assets to be modified at runtime.

 

The Storefront application's Themes feature (see Themes documentation) required the introduction of a StoreResourceManager to handle Velocity's lookups and caching for Storefront velocity templates. It does not use the AssetRepository, but instead uses very similar code combined with hardcoding the velocity template directory within the assets folders (see StoreVelocityConfigHelper.java). A separate Velocity Engine is created for each and every Storefront operating within a WAR.

The Storefront's asset requests that are not for Velocity templates (e.g. css, javascript, images, Power Reviews) go through the AssetResourceController, which uses the AssetRepository indirectly through an AssetRetrievalStrategy. In order for controllers to know which store they are working with, a ThreadLocal reference to Store has been introduced, set by the StoreSelectionFilter. Any storefront controller requiring parameters specific to the store that is accessing the controller can inject the StoreConfig object, or they can retrieve it from the RequestHelper, which also has it injected. The current store is no longer retrieved from ElasticPath.java.

4. New DAO classes

New DAO classes were introduced in some areas to separate service logic from persistence operations. In addition to some new features using separate Service and DAO classes, existing Services which have had DAO functionality factored out include:


  • ProductService / ProductDao
  • ProductTypeService / ProductTypeDao

 

DAO classes should generally not be accessed directly from outside of the Service layer. Such separation of business logic and persistence logic allows for far more flexibility and decoupling in the application. Future development should endeavor to follow this pattern (See the GiftCertificateThemeService / GiftCertificateThemeStoreAssignmentDao for an example of how such decoupling can be of benefit).

5. Index UID / GUID fields

We are phasing out UID fields in the indexes, in favor of GUID fields, for greater decoupling. Secondary object UID fields have, for the most part, been replaced with GUID fields in object indexes so that searches can be performed based on GUIDs. For instance, the CategoryIndex contains the CategoryUid so that the UID can be retrieved from the index when a search match is found, but the index is built using the GUIDs from associated objects (e.g. you can search the CategoryIndex to return categories that are associated with a particular CatalogCode, but not with a particular CatalogUid). The corresponding SearchCriteria objects have been modified to reflect the index schema changes.

Index fields that have been modified as a result of this change are as follows:


  • CategoryIndex:

 

  • parentCategoryUids replaced with parentCategoryCodes
  • catalogUid replaced with catalogCode

 

  • PromotionIndex

 

  • promotionRuleSetUid continues to live in the index, but the promotionRuleSetName is now also in the index. This is for legacy reasons (the PromotionSearchCriteria object still contains both fields as well), and the UID field is expected to be removed in a future version of Elastic Path.
  • catalogUid continues to live in the index, but catalogCode is now also in the index. This is for legacy reasons (the PromotionSearchCriteria object still contains both fields as well), and the UID field is expected to be removed in a future version of Elastic Path.

 

  • ProductIndex

 

  • CategoryUid replace with CategoryCode
  • parentCategoryUids replaced with parentCategoryCodes
  • catalogUid replaced with catalogCode
  • Addition of a new storeCode field

 

In addition, several fields were added to the OrderReturn index schema.

6. Improved Index Build Status Tracking

The status of indexes, and the method of causing them to rebuild, has changed drastically:


  • The index build status (including the last time the index was built) is stored in the new TINDEXBUILDSTATUS database table, and can be accessed using the IndexBuildStatusService.
  • On the Search Server, the AbstractIndexServiceImpl class manipulates the status of index builds directly through the IndexBuildStatusDao.
  • The Commerce Manager Client has a new View that enables index rebuilds to be triggered manually.

 

In addition, if the server is brought down (either manually or catastrophically) while in the process of building an index, it will now remember its status upon restart and will start a new rebuild rather than assuming that the index is complete.

7. New Store Editor GUI

The Store Editor has been enhanced to better handle store themes. In addition, it has abstracted most of the business logic from where it may previously have been embedded in the UI code.

0 Comments 0 References Permalink

Elastic Path 6.1 provides a new way to manage your application configuration. In the past, settings were stored in a variety of places throughout the system; some were in XML files, others were in properties files, and still others were stored in different tables in the database. Now, the most commonly used settings are stored in the database and can be configured using the Commerce Manager.

Custom settings can be migrated to use this framework with little code change and would be able to take advantage of Elastic Path 6.1 multi-store setting management features.

Setting Definitions

Setting definitions are stored in the database, in the TSETTINGDEFINITION table. A setting definition contains:

 

  • a path
  • a default value
  • a value type
  • the maximum number of overrides

 

The path is a unique identifier for the setting. For example COMMERCE/STORE/giftcertificatesEnabled path refers to the setting that sets whether gift certificates are enabled for a given store.

The default value is the fallback to use when a specific value hasn't been assigned to the setting.

 

The value type indicates the type of value stored in the setting (a string, a number, a URL, etc.).

 

The maximum number of overrides specifies the number of values that can be assigned to a setting. Some settings can have multiple values assigned to them. Others can only have one. For example, a store-specific setting would need to have different values for different stores, so the number of overrides would be very large or unlimited (-1). For a system-wide setting, there is generally only one override.

Setting Values

The TSETTINGVALUE table contains the values that have been assigned to settings. Each value has a context. For system settings, the context is empty. For store-specific values, the context contains a store code.

Setting Metadata

Setting metadata is stored in the TSETTINGMETADATA table. It can be used to store additional configuration information relating to the setting. A setting can have any number of metadata associated with it.

Setting metadata consists of a set of key/value pairs. In Elastic Path 6.1, there are two metadata key/value pairs that can be assigned to settings:

 

  • RefreshStrategy, which defines the setting's refresh strategy

 

  • availableToMarketing, which indicates whether the setting is displayed in the store Marketing tab.

 

Setting Refresh Strategies

The setting refresh strategies metadata provides fine-grained control over when setting changes are applied to the system. In the past, most setting changes required restarting the server. Now, when the setting change is applied depends on how you've configured its refresh strategy. Out of the box, Elastic Path 6.1 provides four refresh strategies:

 

  • application: settings that use the application refresh strategy are only applied after the application is restarted

 

  • session: settings that use the session refresh strategy do not change during the lifetime of the user's session. If an administrator changes the setting value, users will not see the change until their session cookie is removed or expires and a new session is created

 

  • interval: settings that use the interval refresh strategy are cached for a specified period of time. Users do not see changes until the cache expires

 

  • immediate: settings that use the immediate refresh strategy are updated immediately. As soon as the administrator changes a setting that uses this strategy, the change is applied everywhere

Creating a Setting Definition

New setting definitions can be added by executing SQL insert statements. For example, the following creates a definition identified by the path CUSTOM/WIDGETS/widgetType:

INSERT INTO TSETTINGDEFINITION(UIDPK, PATH, DEFAULT_VALUE, VALUE_TYPE, MAX_OVERRIDE_VALUES)
VALUES(10001, "CUSTOM/WIDGETS/widgetType", "flat", "String", 0, 1);

 

For your custom settings, you should use a naming convention, to avoid conflicts with other settings. Do not use the COMMERCE path prefix, which is reserved for Elastic Path settings.

Adding a Value for the Setting

Setting values can be added by using the Commerce Manager client application. For more information, see the Commerce Manager User Manual.

You can also use SQL insert statements. Remember that you don't need to create a setting value if there is a default value that comes from the setting definition.

Assuming you do want to override the setting definition for a specific context (store code for stores), you can create the value with the following SQL:

INSERT INTO TSETTINGVALUE(UIDPK, SETTING_DEFINITION_UID, CONTEXT, CONTEXT_VALUE)
 VALUES(1011, 10001, "SLRWORLD", "circle")

 

Note the value of SETTING_DEFINITION_UID should match the UIDPK for the related row in TSETTINGDEFINITION.

Note that multiple setting values can be defined on a setting definition if the setting definition's max_override_values field is set to -1.

 

Settings Framework Services

Elastic Path 6.1 provides a low level SettingsService class to manage and retrieve of setting definitions and values. To get a setting value for a system level setting with no context:

SettingsService settingsService = getElasticPath()
   getBean(ContextIdNames.SETTINGS_SERVICE);
final int numOfMinutes = Integer.valueOf(settingsService.getSettingValue(
   "COMMERCE/APPSPECIFIC/RCP/idleTimeForLock").getValue());

 

To get a setting value for a particular context:

settingsService.getSettingValue(path, store.getCode()).getValue();

 

If there is no setting value in TSETTINGVALUE for the specified context, the system will fall back to the default value in TSETTINGDEFINITION. For example, the default value of the COMMERCE/STORE/giftcertificatesEnabled setting is false, but a setting value for the context SNAPITUP is true, so the Snap It Up store has gift certificates enabled.

 

A SettingsReader service interface is provided for read only operations. Additionally, the CachedSettingsReader service wrapper provides caching for settings retrieval, and is the preferred method in the storefront to avoid frequent database calls.

 

In storefront controllers, the StoreConfig object (available through the RequestHelper) can be used to obtain setting values. If a store specific value is available, it will automatically be retrieved. Otherwise, the default value from the setting definition is returned.  For example:

String pagination = getRequestHelper().getStoreConfig()
 getSetting("COMMERCE/STORE/CATALOG/catalogViewPagination").getValue();
int paginationNumber = NumberUtils.toInt(pagination, DEFAULT_PAGINATION);

 

Elastic Path 6.1's settings framework should replace usage of configuration files for custom configuration settings, especially where multiple stores are being deployed. Previous code retrieving settings from the ElasticPathImpl object should be migrated to the settings framework and its suite of APIs.

0 Comments 0 References Permalink

       

The search server web application is responsible for providing searching and browsing functionality for both the storefront and the Commerce Manager (CM) client. At its heart is the Solr search engine, which sits on top of the Lucene indexes. If you wanted to leverage Solr's powerful search capabilities, you could use the Solr search APIs. However, these APIs require extensive knowledge of Solr syntax. Another option might be to use JPQL, which is basically an object oriented SQL. But this requires some JPQL specific knowledge to be able to retrieve data from the database.

 

In Elastic Path 6.1, a new set of APIs were created to allow developers to create complex, platform-independent search queries using a familiar syntax, but with the added benefits of an ecommerce domain-specific language.  The new Elastic Path Query Language (EPQL) gives us the ability to be technology independent and makes the system more flexible.

 

The advanced search feature was introduced in Elastic Path Commerce 6.1, with the ability to handle products, categories, and catalogs initially. Its architecture, however, is extensible, and in the future, it will support searching for other types of domain objects, such as price lists, orders, and customers.  The advanced search functionality is currently available within the CM client and the import-export tool. The CM client integration supports only products at this time, while the import-export tool can use advanced search for looking up sets of products, categories, and catalogs.

 

Advanced search is provided as a library that can be used into any application or a standalone program.

 

The syntax of the Elastic Path Query Language (EPQL) resembles the SQL language. A simple query consists of a single expression. An expression has the following form:

 

FIND Product WHERE <field> <operator> <value>

where

  • <field> is the field you are searching. For example, if you want to look for products of a specific brand, you would include the BrandCode field in your query. The supported fields are described in Supported fields further in this article.
  • <operator> is the operator you are using to perform the comparison.
  • <value> is the literal value you want to compare to the field value.

For example, the following query matches the product whose code is 10030205

 

FIND Product WHERE ProductCode =   '10030205'

 

In addition to searching for field values, you can also search for attribute values. To search for a value in an attribute, the expression has the following form:

 

Attribute{<attribute_name>} <operator> <value>

where <attribute_name> is the name of a product attribute or product SKU attribute.

 

For example, the following query matches all products that have the Header / Model attribute set to MX:

 

FIND Product WHERE Attribute{Header /   Model} = 'MX'

 

Here is a more complicated query that finds all Nike and Adidas products that cost less than $200 USD and belong to catalog A:

 

FIND Product WHERE Catalog = 'A'   AND Price{A}[USD] < 200

AND (BrandName [en] = 'Nike' OR BrandName   [en] = 'Adidas')


You can run queries immediately using the CM client's UI. This is convenient for testing, but it doesn't allow you to schedule actions on the query results. Let's say you want to perform a daily action on products that match the criteria in the previous example. To do that, you would need to create a class that retrieves the products using EPQL and sends that information to a third party system for processing.

 

To use the EPQL search APIs, you need to add com.elasticpath.core.jar and com.elasticpath.ql.jar file to your project's classpath.

 

Next, add the required Spring bean definitions. Create a file named serviceEPQL.xml in the project's conf/spring/service folder and add a reference to it from the Spring application-context.xml. For an example, take a look at the com.elasticpath.cmclient.core RCP plugin or the import/export tool.

 

Create a Java class that will be used to retrieve the products using EPQL. The following code shows how to execute an EPQL query:

 

public boolean doDailyTask() {

// get a search engine instance

EpQLSearchEngine searchEngine =

       getElasticPath().getBean("epQLSearchEngine");

// create a query

String searchString = "FIND Product   WHERE Catalog = 'A' " +

       "AND Price{A}[USD] < 200   AND (BrandName [en] = 'Nike' " +

       "OR BrandName [en] =   'Adidas')";

// get a parser and use it to validate the query

EPQueryParser parser = searchEngine.getEpQueryParser();

try {

   parser.verify(query);

}

catch (EpQLParseException   exception){

   LOG.warn(exception);

return false;

}

// if the query is okay, execute it and   get the results

SolrIndexSearchResult result =   searchEngine.search(searchString);

List<Long> productUids =   result.getSearchResults();

// ... Do some processing on the results

return true;

}

 

The EpQLSearchEngine class has a search method that executes a query and returns the results. Before executing a query, you should make sure that it is syntactically valid.

 

The EPQLQueryParser class provides a verify method that takes a query string as an argument. If there's an error in the query, it throws an exception, including a detailed description to help you fix the problem.  The search result set contains the list of UIDs of the objects that matched the query.

0 Comments 0 References Permalink

       

The search index rebuild process has undergone some substantial changes in Elastic Path 6.1, making it more robust and more usable.

 

In the past, to trigger a search index rebuild, you needed to manually modify the contents of some properties files on the search server. This is no longer supported. Now, Commerce Manager users with the proper permissions can initiate search index rebuilds.

 

In the Activity menu, choose Configuration. In the Configuration page on the left, under System Administration, click Search Indexes.

 

Select an index in the list, and click Rebuild Index. The rebuild should now be scheduled and the status column will show Rebuild is Scheduled.

When the index rebuild begins, the status column for the index being rebuilt will contain Rebuild in Progress. Note that you can rebuild multiple indexes at the same time. When the rebuild is done, the status is changed to Complete.


When the user runs an index rebuild from the Commerce Manager client, this is what happens behind the scenes:

 

  1. The TINDEXNOTIFY table in the database contains a row for each search index. When the user clicks Rebuild Index, the UPDATE_TYPE column for that search index's row is set to REBUILD.
  2. The quartz.xml in com.elasticpath.search/WEB-INF/conf/spring/scheduling defines several Quartz jobs that check to see if the search indexes need to be rebuilt. Each time one of these scheduled jobs is triggered, it calls the method buildIndexJobRunner method in the AbstractIndexService and passes it a search index identifier. This method checks the TINDEXNOTIFY database table to see if the corresponding index needs to be rebuilt (if the value is REBUILD), has never been built, or if a previous rebuild was interrupted and did not complete successfully. If any of those conditions are true, a rebuild of that index is initiated.
  3. When an index rebuild is initiated, the INDEX_STATUS column in the TINDEXBUILDSTATUS table is set to REBUILD_IN_PROGRESS.
  4. After the index has finished rebuilding, the INDEX_STATUS column is set to COMPLETE.

 

You can monitor the status of rebuild process by checking the Status column in the search index list.

 

Finally, search index rebuilds are now more resilient to failures. If an index build is interrupted or fails to complete normally, the system will try to build it again on server startup or at the start of the next quartz job. If there is a rebuild in progress for an index and another rebuild for that index is requested by the user, the system will wait until the first rebuild is complete before processing the second request.

0 Comments 0 References Permalink

     

Elastic Path Commerce 6.1 introduces a number of changes from version 6.0.x. Primary among these are a major change to the store asset directory structure; movement of store configuration information from the file system to the database; and a number of minor changes to the existing database tables. As a result, 6.0.4 deployments with existing data can be upgraded to the 6.1 codebase with relocation of store assets; migration of configuration files into the database; and database upgrade scripts.

 

Note that it's assumed that the codebase has already been upgraded to 6.1; it's up to you to decide how code merge is done. It's also assumed that the existing deployment was already at the latest service release (6.0.4) before upgrading to 6.1.


Store Assets Relocation

Along with the XML configuration files, store-specific visual presentation elements used in the Storefront and emails have been moved from the WAR files into a central assets directory.  These include all Velocity templates, message properties files, images, javascript, css and Power Reviews.

 

  • The simplest way to move storefront assets out of the WAR file is to copy the directories com.elasticpath.sf/template-resources and com.elasticpath.sf/WEB-INF/templates to the new assets directory under a theme folder of your choosing. The assets directory will have the structure:

 

assets/themes/<theme_name>/<store_code>/templates

assets/themes/<theme_name>/<store_code>/template-resources

 

  • Email templates previously located in com.elasticpath.cm/WEB-INF/templates also need to be moved to the assets folder. Email templates and property files that are not connected to any particular store will need to be moved to: assets/cmassets/templates/velocity

 

  • Storefront related emails will be located under their respective theme folders: assets/themes/<theme_name>/<store_code>/templates/velocity

Configuration File Migration

Much of the configuration data previously in XML files within Elastic Path WAR directories has been moved into the database to be managed through the Settings framework in the CM. This includes commerce-config.xml, search-config.xml, urlrewrite.xml, intelligent-browsing.xml. Please consult the settings framework documentation before migrating configuration settings.

 

  • With the exception of web.sf.context.url, all settings previously in commerce-config.xml have been moved to the Settings framework and are no longer required to be in the XML file. Values in these settings (e.g. VFS paths/login) should be updated in the Settings framework through the CM Settings configuration view as new Setting values or inserted into tables TSETTINGVALUE where appropriate. Additional custom settings in commerce-config.xml file should be moved to the Settings framework as new Setting definitions in TSETTINGDEFINITION. A table of where settings in the commerce-config.xml reside in the settings framework is provided in Appendix A.

 

  • Content in intelligent-browsing.xml has been moved to the Settings framework and the file is no longer needed. Its contents should be inserted as a new Setting Value for the setting COMMERCE/STORE/FILTEREDNAVIGATION/filteredNavigationConfiguration

 

The content can also be copied into the CM Store Editor's Filtered Navigation section.

 

  • Content in urlrewrite.xml has been moved to the Settings framework and the file is no longer needed. Its contents should be inserted as a new Setting Value for the setting COMMERCE/SYSTEM/urlRewriteConfig

 

  • Content in search-config.xml has been moved to the Settings framework and the file is no longer needed. If any customization has been made, the Setting values under the paths COMMERCE/SEARCH/<setting> should be updated appropriately.

Database Upgrade

Upgrade SQL scripts are provided along with each release of Elastic Path to capture changes in the database schema between the new release and the latest service release.

Each supported database (MySQL, SQL Server, Oracle) will have its own upgrade script.

 

  • Upgrade scripts are provided in the 6.1 distribution for each database, and should be run by the tools provided by the database vendor.
  • Upgrade scripts will apply schema changes and insert necessary base data on an existing 6.0.x database, but does not change existing data.

 

After running the upgrade script, the steps can be done through the CM or directly through the database to get your deployment up and running:

 

  • Reassign supported currency and supported locales to existing stores in the CM, or insert entries in the TSTORESUPPORTEDCURRENCY and TSTORESUPPORTEDLOCALE respectively.
  • Change the display theme of existing stores appropriately by editing the COMMERCE/STORE/theme setting in either the System Configuration section of the CM or the Store Editor.
  • Set the correct value for the search server in the settings framework, under the path COMMERCE/SERVER/SEARCH/searchHost
  • Search indexes need to be rebuilt. This is an automatic step if old indexes are deleted.
  • Set the correct values for the asset VFS in the settings framework under COMMERCE/APPSPECIFIC/RCP/ASSETS/<setting>
  • Gift certificate themes need to be recreated through the CM, and assigned to stores in the CM Store Editor.
  • Be aware that stores can now only be accessed through the URL defined in the CM Store Editor (TSTORE table in the databse), and care should be taken when testing.

Caveats

  • The upgrade process cannot cover all possible database and code customizations to the 6.0.x codebase, and upgrade scripts should be reviewed before running on any schemas that you have customized.
  • Before moving existing storefront assets, be aware that there are changes in these files (Velocity templates, property files) that might not be picked up through a normal code merge due to the change in location.
  • New configuration settings introduced in 6.1 take on their default values after an upgrade, and should be reviewed for correctness.

 

Upgrading to 6.1 from 6.0.4 requires little database change. Configuration files and asset migration make up the bulk of the upgrading effort, and existing customizations to configuration and assets should be reviewed to take advantage of the new settings framework and asset repository.

0 Comments 0 References Permalink

In Elastic Path 6.1, we updated some of our library dependencies to more recent versions. This article outlines the changes, how they impact the application in terms of performance and behaviour, and how they might impact any custom Elastic Path development work you might be doing.

 

  • JUnit has been upgraded from version 3.8.1 to 4.4. Version 4.4 of the JUnit testing framework provides more powerful unit testing features, leveraging Java annotations. The upgrade is fully backwards compatible with tests written against older versions of JUnit. For more information on JUnit 4, go to http://junit.org/.
  • jMock has been upgraded from version 1.1 to 2.5. jMock 2.5 provides better management and creation of mock objects, but is not fully compatible with version 1.1, so both versions still exist as dependencies in Elastic Path 6.1. For more information on jMock 2.5, go to http://www.jmock.org/.
  • For an example of a test class in that uses the features of JUnit 4 together with jMock 2, see com.elasticpath.service.impl.GiftCertificateThemeServiceImplTest in the com.elasticpath.core product.#
  • The Spring framework was upgraded from 2.0.4 to 2.5.5. This upgrade brings improved performance, ease of configuration, and new features. For more information on the changes, see the Spring framework site (http://static.springframework.org/spring/docs/2.5.x/reference/new-in-2.html).  This upgrade is backwards compatible with previous versions of the Spring framework and customizations should not be affected.
  • Url Rewrite Filter was upgraded from 2.5.2 to 3.1. The newer version includes support for wildcard matching, mod_rewrite style configuration, and others new features.  This upgrade is backwards compatible with previous versions and customizations should not be affected. The complete changelog can be found at the Url Rewrite Filter site:http://urlrewritefilter.googlecode.com/svn/trunk/src/doc/manual/3.1/introduction.html#changelog
0 Comments 0 References Permalink

      

Introduction

Asset management has changed significantly in EP 6.1 to make deployment of multiple stores easier. With EP 6.1, multiple stores may be run in a single WAR file, hence there is no need to copy WAR instances to deploy a new store. Asset types, which include images, CSS, JavaScript, email templates, storefront velocity templates, VM_global_libraries and Message Resource property files, have been moved out of the WAR deployment location and into a new directory elsewhere on the file system. The previous structure of the assets directory has also changed.

 

The assets root directory structure is as follows:

 

- assets

+ cmassets

+ digitalassets

+ images

+ import

+ storeassets

+ themes

 

The absolute path for assets is specified in your deployment's COMMERCE/SYSTEM/ASSETS/assetLocation setting. EP6.1 also introduces the "Settings Framework," a new component that contains system and store configuration settings that used to be captured in commerce-config.xml and other files. The COMMERCE/SYSTEM/ASSETS/assetLocation setting value must be configured before you start your storefront application to point at your assets root directory, for example c:\assets or /opt/assets.

 

  • The cmassets directory is also new in EP 6.1 and contains the Commerce Manager specific files, such as user account email templates and the import report email template.
  • The digitalassets directory has been around for several releases and still contains the downloadable digital product files.
  • The images directory is also not new and contains your product display images.
  • The import directory was introduced in EP 6.0 and contains all import CSV files used by the Commerce Manager. Please note that these files are not used by the Import/Export tool, which is a separate component.
  • The storeassets directory is new in EP 6.1 and only contains the PowerReviews required by each store. This directory is specific to each store, but does not form part of a theme.
  • The themes directory is new and serves as the root directory for all themes used by your stores. Themes are used to define the look and feel of stores in EP. You can create multiple themes and associate these with one or more stores. You can also customize theme elements (images, CSS, JavaScript) for individual stores without affecting other stores that use the same theme.

 

 

A theme consists of a set of Velocity templates and resources required to display a storefront (stylesheets, images, properties files, etc.). Each theme is stored in its own directory under the assets/themes directory:

 

- assets

+ cmassets

+ digitalassets

+ images

+ import

+ storeassets

- themes

  + electronics

  + mytheme1

  + mytheme2

  + ...

 

Each theme must contain a default directory. This directory contains the templates and the template-resources directories. The templates directory contains the Velocity templates used by the Velocity Engine to render HTML pages. The template-resources directory contains the stylesheets, Javascript and image files referred to by the rendered HTML pages.

 

- assets

+ cmassets

+ digitalassets

+ images

+ import

+ storeassets

- themes

  - electronics

    - default

      - template-resources

        + images

        + js

        + stylesheet

      - templates

        - velocity

          + index.vm

Within these directories, you can use any directory structure you want to organize files, but keep in mind the file-based fallback mechanism used within themes.

 

 

Themes can be customized to suit the needs of individual stores. To customize a theme for a specific store, create a directory under the theme and set the directory name to the store code. For example, to customize mytheme1 for the SNAPITUP store, create a directory named SNAPITUP under mytheme1. The directory structure should look similar to the following:

 

- assets

+ themes

- mytheme1

  + default

  + SNAPITUP

 

In the store-specific directory, only create copies of the files that you want to customize. The copies must exist in the same relative directory structure within the store-specific directory. For example, to customize the mytheme1/default/template-resources/images/image-not-available.png image for the SNAPITUP store, place the custom version of image-not-available.png in mytheme1/SNAPITUP/template-resources/images.

 

- assets

- themes

  - mytheme1

    - default

      - template-resources

        - images

          + image-not-available.png (default image)

          + ... all other shared images

    - SNAPITUP

      - template-resources

        - images

          + image-not-available.png (store overriding image)

 

It is not necessary to copy the entire contents of the default directory into the store-specific directory. You only need to include copies of the files that you are customizing. The application looks for files in the store-specific directory first; if it does not find a file in the store-specific directory, it then looks for it in the default directory. In other words, your customized copy of the file or image acts as an override for the default file or image. This file-based overriding mechanism is used for images, stylesheet, JavaScript and Velocity template files.  Access to these resources is handled by resource controllers, which in turn use resource retrieval strategies for greater flexibility.

 

WEB-INF/conf/spring/web/url-mapping.xml

<bean id="SimpleUrlHandlerMapping"

    class="org.springframework.web.servlet.handler.SimpleUrlHandlerMapping">

  <property name="mappings">

      <props>

      <prop key="/images/**/*">storeThemeAssetResourceController</prop>

      <prop key="/stylesheet/**/*">storeThemeAssetResourceController</prop>

      <prop key="/js/**/*">storeThemeAssetResourceController</prop>

      </props>

  </property>

</bean>

<!-- Resource controllers for resource retrieval -->

<bean id="storeThemeAssetResourceController"

    class="com.elasticpath.sfweb.controller.impl.AssetResourceControllerImpl"

    parent="abstractEpController">

  <property name="resourceRetrievalStrategy"

            ref="storeThemeAssetRetrievalStrategy"/>

</bean>

Message Sources use a property based overriding mechanism, as opposed to the file based overriding for other types of assets like images. Message Sources are a collection of property files, which associate message keys to message text in different languages. Take, for example, the globals.firstname property key that's located in 2 files, one referencing the English message and the other the French message.

- assets

- themes

  - mytheme1

    - default

      - templates

        + globals.properties  : globals.firstname=First Name

        + globals_fr.properties: globals.firstname=Prenom

 

To override the French globals.firstname message for the SNAPITUPUK store, one need only override one property, not the entire property file. In this case the global_fr.properties file need only contain a single property entry for globals.firstname.

 

- assets

- themes

  - mytheme1

    - default

    - templates

        + globals_fr.properties: globals.firstname=Prenom (default)

    - SNAPITUP

      - templates

        + globals_fr.properties: globals.firstname=Prénom (overriding)

 

In fact, the file name need not even start with "globals" as all properties are loaded into a collection at which point file names are no longer relevant. The only requirement is that it contain the language code. For example, having the globals.firstname property override in a file named no-name-in-particular_fr.properties, will have the same effect. If no language code is present, English is assumed.  For reference, see the Message Source configuration in the Storefront and Commerce Manager web applications.

 

WEB-INF/conf/spring/views/velocity/velocity.xml

<bean id="messageSource"

    class="com.elasticpath.commons.util.impl.StoreThemeMessageSource" >

  <property name="messageSourceCache">

      <ref bean="messageSourceCache"/>

  </property>

  <property name="storeConfig">

      <ref bean="threadLocalStorage"/>

  </property>

  <property name="globalMessageSource">

      <ref bean="globalMessageSource"/>

  </property>

  <!-- True if you want missing messages to show up

        as just the key used to look for them. e.g. user.firstname.

        False will allow exception to be thrown, and stop template render.

  -->

  <property name="useCodeAsDefaultMessage">

      <value>true</value>

  </property>

</bean>

 


Using Store Editor

 

You can select the theme using the new store editor:

1.  In Commerce Manager, choose *Activity*->*Configuration*.

2.  Under *Stores*, click *Stores*.

3.  Select the store, then click *Edit Store*.

4.  Click the *Theme* tab.

5.  In the _Theme_ box, enter the name of the theme folder. For example, if the theme folder is C:/Program Files/Apache Software Foundation/apache-tomcat-5.5.26/assets/themes/mytheme1, enter mytheme1.

6.  In the Commerce Manager toolbar, click *Save*.


Using Advanced Setting Configuration

 

By default, the demo store is mapped to the electronics theme. To change the theme of a store:

 

1.  In Commerce Manager, choose Activity -> Configuration.

2.  Under System Administration, click System Configuration.

3.  In the Property Name list, select COMMERCE/STORE/theme.

4.  Click New (next to the Defined Values list).

5.  Set the values as follows:

·        Context: The store code. For example, for the SnapItUp demo store, the store code is SNAPITUP.

·        Value: The name of the theme folder. For example, if the theme folder is C:/Program Files/Apache Software Foundation/apache-tomcat-5.5.26/assets/themes/mytheme1, enter mytheme1.

6.  Click Save.

 

 

During the development of EP 6.1 it was clear that we had to find a way to make the Velocity engine's VM_global_library store specific since stores do not always share the same library macros.  In EP 6.0, Spring instantiated a VelocityEngine bean directly to inject into the onePageHelper bean, as OnePageHelperImpl requires a Velocity engine for some functionality. This bean is also injected into the velocityConfig bean which is what the VelocityView class uses to get the Velocity engine (it is effectively a factory). The VelocityView class retrieves the Velocity engine once, but we want it to get the appropriate store engine each time the view is asked to render a template.

 

Among other smaller changes, the following classes needed to be added in EP 6.1 to correct the Velocity Engine behavior.

 

<bean id="viewResolver"

class="com.elasticpath.sfweb.view.velocity.StoreVelocityViewResolver">

  <property name="viewClass">

<value>com.elasticpath.sfweb.view.velocity.StoreVelocityViewImpl</value>

  </property>

  <property name="velocityConfigurer" ref="velocityConfigurer"/>

  ...

</bean>

<bean id="velocityConfigurer"

    class="com.elasticpath.sfweb.view.velocity.StoreVelocityConfigurer">

  <property name="storeConfig" ref="threadLocalStorage"/>

  <property name="velocityPropertiesMap" ref="velocityPropertiesMap"/>

</bean>

 

StoreVelocityConfigurer

 

This is a subclass of VelocityConfigurer which keeps a Map of velocity engines keyed by store code. This class just overrides the getVelocityEngine() method and uses an injected store config to retrieve the appropriate VelocityEngine instance from the map. It will call createVelocityEngine() if there is no value for the store currently in the map.

 

StoreVelocityViewResolver

 

This is a subclass of VelocityViewResolver which is store aware. It has setters for a StoreConfig and VelocityConfigurer which will typically be injected by Spring. This class overrides the buildView method and passes the store config and velocity configurer through to the StoreVelocityViewImpl class so it can render the store-specific view.  The old "global" WEB-INF/VM_global_library.vm library has been removed. You now have a VM_global_library in the templates/velocity directory of a theme or store-specific theme override. This will fall back exactly like the other assets.

 

- assets

- themes

  - mytheme1

    - default

      - templates

        - velocity

          + VM_global_library.vm (default fall-back)

    - SNAPITUP

      - templates

        - velocity

            + VM_global_library.vm (store specific override)

 

Store Specific Report Templates

 

Report Design files (*.rptdesign) are typically bundled as part of the Commerce Manager plugin. Packing Slip reports are store specific though, and had to be moved out of the plugin and uses the theme to take advantage of the file-based overriding mechanism.

 

- assets

- themes

  - mytheme1

    - default

      - templates

        - printartifacts

          + packing_slip.rptdesign (default report design)

    - SNAPITUP

      - templates

        - printartifacts

            + packing_slip.rptdesign (store specific override)

 

 

The com.elasticpath.cmweb.controller.impl.CacheInvalidationControllerImpl defined in the CM Server is useful to invalidate caches of various types based on an invalidation strategy. The default configuration, as displayed below and duplicated in the Storefront, uses the TriggeredCacheInvalidationStrategy to invalidate the Velocity engines and message resources.

 

com.elasticpath.cm/WEB-INF/conf/spring/web/url-mapping.xml:

<prop key="/invalidate-cache.ep">cacheInvalidationController</prop>

<bean id="cacheInvalidationController"

    class="com.elasticpath.cmweb.controller.impl.CacheInvalidationControllerImpl">

  <property name="cacheInvalidationStrategy"

            ref="triggeredCacheInvalidationStrategy" />

</bean>

com.elasticpath.cm/WEB-INF/conf/spring/service/serviceCM.xml:

<bean id="triggeredCacheInvalidationStrategy"

class="com.elasticpath.commons.util.impl.TriggeredCacheInvalidationStrategyImpl">

  <property name="invalidatableCaches">

      <bean class="java.util.HashSet">

          <constructor-arg>

              <set>

                  <ref bean="velocityEngineFactory"/>

                  <ref bean="messageSourceCache"/>

              </set>

          </constructor-arg>

      </bean>

  </property>

</bean>

Be sure to have the invalidation URL secured, as it will cause a hit in performance every time the velocity engine is recreated, which reloads all the message property files.

 

 

For information on configuring PowerReviews in Elastic Path, see the PowerReviews Wiki page at https://wiki.elasticpath.com/display/EP61DEPLOY/5+-+PowerReviews+(optional) in the Deployment Guide. For information on the Power Reviews system, see the PowerReviews website at http://www.powerreviews.com.  The Power Reviews component is enabled and disabled by the boolean store-specific setting COMMERCE/STORE/POWERREVIEWS/powerreviewsEnabled. The value of the setting is passed to Velocity code by the powerReviewsEnabledHelper Spring bean defined in com.elasticpath.sf/WEB-INF/conf/spring/service/serviceSF.xml. Power Reviews are rendered by JavaScript code, and examples can be found in the guidedSkuTemplate.vm and productTemplate.vm templates.

 

WEB-INF/conf/spring/web/url-mapping.xml

<bean id="SimpleUrlHandlerMapping"

    class="org.springframework.web.servlet.handler.SimpleUrlHandlerMapping">

  <property name="mappings">

      <props>

      <prop key="/power-reviews/pwr/engine/js/**/*">

            storeAssetResourceController</prop>

      <prop key="/power-reviews/pwr/engine/**/*">

            storeAssetResourceController</prop>

      <prop key="/power-reviews/pwr/content/**/*">

            storeAssetResourceController</prop>

      </props>

  </property>

</bean>

<!-- Resource controllers for resource retrieval -->

<bean id="storeThemeAssetResourceController"

    class="com.elasticpath.sfweb.controller.impl.AssetResourceControllerImpl"

    parent="abstractEpController">

  <property name="resourceRetrievalStrategy"

            ref="storeThemeAssetRetrievalStrategy"/>

</bean>

<bean id="storeAssetResourceController"

    class="com.elasticpath.sfweb.controller.impl.AssetResourceControllerImpl"

    parent="abstractEpController">

  <property name="resourceRetrievalStrategy"

            ref="storeResourceRetrievalStrategy"/>

</bean>

0 Comments 0 References Permalink

There were a number of changes to the database schema in Elastic Path 6.1 compared to the schema in Elastic Path 6.0.4. Some new tables and columns were added to support new functionality, one table that is no longer used was removed, and a few columns were modified to increase data storage efficiency and data integrity.

 

Let's look at the details of these schema changes.

New Tables for Settings Storage

The settings framework feature was added in 6.1, and settings that were previously stored in commerce-config.xml files in 6.0.x have been moved to the database. The new settings-related tables have interdependencies but are not referentially linked to other tables in the system.

 

  • TSETTINGDEFINITION keeps track of all the application setting definitions and their default values.
  • TSETTINGMETADATA has information on how various settings apply to the system.
  • TSETTINGVALUE keeps track of setting values.

New Table for Advanced Search

TADVANCEDSEARCHQUERY stores saved queries that were created for the Advanced Search feature. It has a referential link to TCMUSER.

New Table for Gift Certificate Themes

TGIFTCERTIFICATETHEME stores gift certificate theme definitions, and has no referential links to any other tables.

New Table for PCI Compliance

TPASSWORDHISTORY stores a user's password history, and has a referential link to TCMUSER.

New Table for Index Build Status

TINDEXBUILDSTATUS stores the status of each SOLR/Lucene index, including its last build date and its current status (built, rebuilding, etc).

New Tables for Storefront Supported Locales and Supported Currencies

TSTORESUPPORTEDLOCALE and TSTORESUPPORTEDCURRENCY were added to take the place of the deleted TSTORE.DEFAULTLOCALE and TSTORE.DEFAULTCURRENCY columns. These tables have a foreign key to the TSTORE table.

Unused Table

TPROMOCODE is a legacy table that is not longer referenced by Elastic Path code and is no longer created by the 6.1 database creation scripts. If you are not using it, it can be safely removed.

New or Modified Columns

The following new columns were added to the schema:

  • Store States
    • TSTORE.STORE_STATE - Integer column - keeps track of the state a store is currently in. Value is required but has a default value of 0.

 

  • PCI Compliance
  • TCMUSER.LAST_CHANGED_PASSWORD_DATE - Date column - value not required.
  • TCMUSER.FAILED_LOGIN_ATTEMPTS - Integer column - value required, defaults to 0.
  • TCMUSER.STATUS - Integer column - value required, defaults to 4.
  • TCMUSER.ENABLED - This column was removed, to be replaced with the STATUS field.


  • Miscellaneous
  • TSTORE.DEFAULT_LOCALE is no longer a required column.
  • TSTORE.DEFAULT_CURRENCY is no longer a required column.
  • TSTORE.EMAIL_SENDER_NAME is no longer a required column.
  • TSTORE.EMAIL_SENDER_ADDRESS is no longer a required column.
  • TSTORE.STORE_ADMIN_EMAIL is no longer a required column.
  • TORDERSHIPMENT.SHIPMENT_NUMBER was increased in size from 10 characters to 64.
  • TINVENTORYAUDIT.EVENT_ORIGINATOR was increased in size from 30 to 255.

 

  • Indexes
  • TORDERLOCK.ORDER_UID is now an indexed column.
  • TORDERSHIPMENT.SHIPMENT_NUMBER is now an indexed column.

 

 

 

So, what does this mean for the average developer working on an upgrade from 6.0.4 to 6.1? Not very much. The database upgrade scripts included in the 6.1 distribution take care of all the necessary schema changes.

 

  • Populating the settings tables with the appropriate values is critical to getting your 6.1 install up and running. For more information, see the Elastic Path 6.1 Upgrade Guide at http://docs.elasticpath.com/display/EP61UPGRADE.
  • Increasing the size of a few columns should have minimal impact.
  • Removing the "required" flag on some fields in the STORE table is essential to the Store States feature, but should have no impact on existing systems.
  • TSTORE.STORE_STATE - 6.0.x only had a single store state: OPEN. The OPEN state in 6.1 is represented by an integer value of 200 in this field.
  • TCMUSER.STATUS replaces the ENABLED column but is backward compatible with its values. The value of the STATUS column should be the same as the value of the ENABLED column for every record during upgrade.
  • The new indexes can be added without any problems.
  • Removing the unused TPROMOCODE table should have no impact, but if you have customizations that are using this table, then it can also be left in with no impact.
  • Creating the new TSTORESUPPORTEDCURRENCY and TSTORESUPPORTEDLOCALE tables for Stores, and creating records corresponding to the column values in the corresponding columns that were removed from TSTORE should be fairly easy.

 

Upgrading an Elastic Path database-from 6.0.x to 6.1 is relatively painless. Only one integer field's data must be translated to a new field and removed (TCmUser.Enabled to TCmUser.Status), and only one new integer field (TStore.store_state) must added and populated with data (but every row has the same value for an upgrade). Other schema changes should have no significant impact on a stock EP6.0.x database.

0 Comments 0 References Permalink