Technical Blog

3 Posts authored by: Tom Manning

The intent of this post is to clear up some foggy notions and misconceptions about the Elastic Path 6.1 development environment, as well as provide a few insider tips and tricks. Developers of CM Client modules will probably find the following more helpful than web or core application developers, but the principles can be used by everyone.


If you're doing any development of the CM Client and you make a change to EP Core, and you want that change to be available in the CM Client, your typical build time shouldn't be more than a minute or so, hopefully much less. Let's clarify:

When you're running the CM Client from within Eclipse, the CM Client will not use the CORE jar; it will use the CORE project's class files directly.
This means that you do NOT have to run "ant all" on the command line (I never run it - it takes forever).

Let's look at a typical day of developing some new feature in the CM Client where you might be changing some code in the EP Core project. If (and only if) you have changed any persistent classes in the core (i.e. domain objects), you have to make sure that JPA has "enhanced" these classes before you can use them, right? So you have two choices (maybe more, but these are the two that I see):

  1. run "ant enhance" in the com.elasticpath.core project and refresh the project in eclipse. This ensures that JPA has a chance to modify the class files that eclipse's compiler created and then eclipse doesn't think its knowledge of the class files is out-of-date. This takes a few seconds.
  2. run "ant build" in one of the web projects (e.g. com.elasticpath.cm), which will rebuild the core project, enhance its class files, and refresh the core jar file that the webapp will use if you're running it outside of eclipse (which most of us don't do anyway - we run it with WTP inside eclipse, so once again the project files are used rather than the core jar file). This takes about 15 seconds on my box (unless maven connections are having problems, but that's a separate issue).


So in a typical developer environment for EP 6.1, using WTP to run the webapps, the CORE jar is never actually used. So it doesn't have to be built, and it certainly doesn't have to be copied anywhere for the CM Client to use it.

Now for some caveats:

  1. If you have previously built the CM Client on the command-line (ant all, ant build-product, etc) you will likely see that it has created a jar file for each plugin in each of the plugin directories. You must delete these or you will find that changes to the code for that plugin are not reflected in the running copy (if it finds a jar, it will use that jar instead of your project code).
  2. The com.elasticpath.ql project seems to be referenced as a jar file from the com.elasticpath.cmclient.libs project. You must build this project once on the commandline so that its jar is created; otherwise you'll find that running the CM Client won't work because the Spring initializer is expecting some code that it cannot find. This often catches developers who have just imported all their eclipse projects and they try to run the CM Client. If you're developing on this ql project you likely already know this; if you're not, then you only have to do this once and you can forget about it.
  3. When setting up a new workspace, you *do* have to do an "ant eclipse-setup" once in the com.elasticpath.cmclient directory to make sure that all the CM Client plugins have eclipse project files generated for them, which you can import into eclipse. The list of projects for which project files will be generated lives in the build.xml file in that directory. If you're adding a new plugin, you will need to add it to that list so that its project file is generated properly (with maven information, etc).
  4. Each Web project requires a META-INF/context.xml file for you to be able to run it in tomcat with WTP. This file is generated by the eclipse-setup script if you've got tomcat specified in your env.config file when you run the script, but I believe it's wiped out when you run "ant clean"... so don't run that, or if you do then make sure you run ant eclipse-setup again.



0 Comments 0 References Permalink

Many classes were refactored in Elastic Path 6.1 to improve testability of existing code and to accommodate new features (most notably the ability for a single Storefront WAR file to host multiple Stores). Here are the changes with the highest impact for developers (in no particular order).

1. Refactoring of ElasticPathImpl.java and Creation of the Settings Framework

ElasticPathImpl covers several distinct areas of responsibility: BeanFactory, Configuration, Filtered Navigation, asset lookup... In 6.1, new interfaces were created to move those areas of functionality out of ElasticPathImpl with the intention of rendering the object obsolete in future versions.

  • Most configuration settings are now accessed via the SettingsService, which will be covered in a separate article. Commerce-config.xml files are no longer used to populate a map of settings in ElasticPathImpl, with the exception of the one in the CM web application, which is only used to keep track of the EP version and build number
  • Asset locations are now retrieved from the AssetRepository interface (see heading elsewhere in this article)
  • Search Configuration is now retrieved entirely from the Settings framework rather than from the ElasticPath object
  • Filtered Navigation information is still retrieved from ElasticPathImpl, but only because it implements the new FilteredNavigationConfiguration interface. The actual retrieval implementation now lives in a separate class and references the new Settings framework
  • Bean instances can still be retrieved from ElasticPath because ElasticPathImpl implements the BeanFactory interface, but from this point onward the ElasticPath object is not required to be injected into classes that only need a BeanFactory. Instead, you should inject an implementation of BeanFactory

If any customizations were performed on some Elastic Path 6.0 code base that must be transferred to an Elastic Path 6.1 code base, most careful attention should be paid to any changes in the ElasticPath singleton, especially if any new settings were introduced that are designed to be read in from a commerce-config.xml configuration file. Since most settings were moved to the Settings framework, the code to parse the configuration file has mostly been removed.

New customizations should pay special attention to the Settings framework and use it where appropriate. Also, the BeanFactory should be injected into new classes where required instead of the ElasticPath object, as its use as a bean factory is being phased out.

New persistent classes should have a DAO that is separate from any new services, to allow for more flexible persistence implementations if functionality changes.

2. Testing Framework Enhancements

In testing, the big news is deprecation of ElasticPathTestCase.java and the introduction of JUnit4 and JMock2 testing frameworks.

Prior to 6.1, many test classes extended ElasticPathTestCase.java, which is a JUnit 3.8 extended test class that mocks out much of the application's domain using mock objects created with the JMock 1.x framework. This class is being phased out, and use of it is no longer recommended because it:

  • is large and unwieldy
  • encourages poor code factoring and integration tests using mock objects
  • is complex; modification may affect many test cases in an unforeseen manner

 

Instead, unit test classes should focus on unit-testing rather than integration-testing methods. Use of the JUnit4 and JMock2 framework features should help in that regard. There are many examples of their use in the new code; but for a start you may want to look at GiftCertificateThemeImplTest (for a relatively simple test class) and StoreResourceManagerImplTest (which takes advantage of some advanced features in the new JMock framework).

3. New Store UI Themes and Asset Retrieval

Store-specific Velocity template resolution has been introduced. Most static and storefront theme content (storefront velocity templates, email velocity templates, javascript files, css files, Power Reviews, etc) was moved outside the separate application WAR files into the pre-existing Assets folder on the file system.

 

In 6.0 only some assets (store images and digital goods) lived on the file system under the assets folder, while others (email templates, storefront UI files, and PowerReviews files) lived inside the WAR. All of these files were moved to appropriate locations under the file system's assets folder and the paths to such folders were made available through an AssetRepository object rather than through methods on the ElasticPath object. This allows the assets to be modified at runtime.

 

The Storefront application's Themes feature (see Themes documentation) required the introduction of a StoreResourceManager to handle Velocity's lookups and caching for Storefront velocity templates. It does not use the AssetRepository, but instead uses very similar code combined with hardcoding the velocity template directory within the assets folders (see StoreVelocityConfigHelper.java). A separate Velocity Engine is created for each and every Storefront operating within a WAR.

The Storefront's asset requests that are not for Velocity templates (e.g. css, javascript, images, Power Reviews) go through the AssetResourceController, which uses the AssetRepository indirectly through an AssetRetrievalStrategy. In order for controllers to know which store they are working with, a ThreadLocal reference to Store has been introduced, set by the StoreSelectionFilter. Any storefront controller requiring parameters specific to the store that is accessing the controller can inject the StoreConfig object, or they can retrieve it from the RequestHelper, which also has it injected. The current store is no longer retrieved from ElasticPath.java.

4. New DAO classes

New DAO classes were introduced in some areas to separate service logic from persistence operations. In addition to some new features using separate Service and DAO classes, existing Services which have had DAO functionality factored out include:


  • ProductService / ProductDao
  • ProductTypeService / ProductTypeDao

 

DAO classes should generally not be accessed directly from outside of the Service layer. Such separation of business logic and persistence logic allows for far more flexibility and decoupling in the application. Future development should endeavor to follow this pattern (See the GiftCertificateThemeService / GiftCertificateThemeStoreAssignmentDao for an example of how such decoupling can be of benefit).

5. Index UID / GUID fields

We are phasing out UID fields in the indexes, in favor of GUID fields, for greater decoupling. Secondary object UID fields have, for the most part, been replaced with GUID fields in object indexes so that searches can be performed based on GUIDs. For instance, the CategoryIndex contains the CategoryUid so that the UID can be retrieved from the index when a search match is found, but the index is built using the GUIDs from associated objects (e.g. you can search the CategoryIndex to return categories that are associated with a particular CatalogCode, but not with a particular CatalogUid). The corresponding SearchCriteria objects have been modified to reflect the index schema changes.

Index fields that have been modified as a result of this change are as follows:


  • CategoryIndex:

 

  • parentCategoryUids replaced with parentCategoryCodes
  • catalogUid replaced with catalogCode

 

  • PromotionIndex

 

  • promotionRuleSetUid continues to live in the index, but the promotionRuleSetName is now also in the index. This is for legacy reasons (the PromotionSearchCriteria object still contains both fields as well), and the UID field is expected to be removed in a future version of Elastic Path.
  • catalogUid continues to live in the index, but catalogCode is now also in the index. This is for legacy reasons (the PromotionSearchCriteria object still contains both fields as well), and the UID field is expected to be removed in a future version of Elastic Path.

 

  • ProductIndex

 

  • CategoryUid replace with CategoryCode
  • parentCategoryUids replaced with parentCategoryCodes
  • catalogUid replaced with catalogCode
  • Addition of a new storeCode field

 

In addition, several fields were added to the OrderReturn index schema.

6. Improved Index Build Status Tracking

The status of indexes, and the method of causing them to rebuild, has changed drastically:


  • The index build status (including the last time the index was built) is stored in the new TINDEXBUILDSTATUS database table, and can be accessed using the IndexBuildStatusService.
  • On the Search Server, the AbstractIndexServiceImpl class manipulates the status of index builds directly through the IndexBuildStatusDao.
  • The Commerce Manager Client has a new View that enables index rebuilds to be triggered manually.

 

In addition, if the server is brought down (either manually or catastrophically) while in the process of building an index, it will now remember its status upon restart and will start a new rebuild rather than assuming that the index is complete.

7. New Store Editor GUI

The Store Editor has been enhanced to better handle store themes. In addition, it has abstracted most of the business logic from where it may previously have been embedded in the UI code.

0 Comments 0 References Permalink

There were a number of changes to the database schema in Elastic Path 6.1 compared to the schema in Elastic Path 6.0.4. Some new tables and columns were added to support new functionality, one table that is no longer used was removed, and a few columns were modified to increase data storage efficiency and data integrity.

 

Let's look at the details of these schema changes.

New Tables for Settings Storage

The settings framework feature was added in 6.1, and settings that were previously stored in commerce-config.xml files in 6.0.x have been moved to the database. The new settings-related tables have interdependencies but are not referentially linked to other tables in the system.

 

  • TSETTINGDEFINITION keeps track of all the application setting definitions and their default values.
  • TSETTINGMETADATA has information on how various settings apply to the system.
  • TSETTINGVALUE keeps track of setting values.

New Table for Advanced Search

TADVANCEDSEARCHQUERY stores saved queries that were created for the Advanced Search feature. It has a referential link to TCMUSER.

New Table for Gift Certificate Themes

TGIFTCERTIFICATETHEME stores gift certificate theme definitions, and has no referential links to any other tables.

New Table for PCI Compliance

TPASSWORDHISTORY stores a user's password history, and has a referential link to TCMUSER.

New Table for Index Build Status

TINDEXBUILDSTATUS stores the status of each SOLR/Lucene index, including its last build date and its current status (built, rebuilding, etc).

New Tables for Storefront Supported Locales and Supported Currencies

TSTORESUPPORTEDLOCALE and TSTORESUPPORTEDCURRENCY were added to take the place of the deleted TSTORE.DEFAULTLOCALE and TSTORE.DEFAULTCURRENCY columns. These tables have a foreign key to the TSTORE table.

Unused Table

TPROMOCODE is a legacy table that is not longer referenced by Elastic Path code and is no longer created by the 6.1 database creation scripts. If you are not using it, it can be safely removed.

New or Modified Columns

The following new columns were added to the schema:

  • Store States
    • TSTORE.STORE_STATE - Integer column - keeps track of the state a store is currently in. Value is required but has a default value of 0.

 

  • PCI Compliance
  • TCMUSER.LAST_CHANGED_PASSWORD_DATE - Date column - value not required.
  • TCMUSER.FAILED_LOGIN_ATTEMPTS - Integer column - value required, defaults to 0.
  • TCMUSER.STATUS - Integer column - value required, defaults to 4.
  • TCMUSER.ENABLED - This column was removed, to be replaced with the STATUS field.


  • Miscellaneous
  • TSTORE.DEFAULT_LOCALE is no longer a required column.
  • TSTORE.DEFAULT_CURRENCY is no longer a required column.
  • TSTORE.EMAIL_SENDER_NAME is no longer a required column.
  • TSTORE.EMAIL_SENDER_ADDRESS is no longer a required column.
  • TSTORE.STORE_ADMIN_EMAIL is no longer a required column.
  • TORDERSHIPMENT.SHIPMENT_NUMBER was increased in size from 10 characters to 64.
  • TINVENTORYAUDIT.EVENT_ORIGINATOR was increased in size from 30 to 255.

 

  • Indexes
  • TORDERLOCK.ORDER_UID is now an indexed column.
  • TORDERSHIPMENT.SHIPMENT_NUMBER is now an indexed column.

 

 

 

So, what does this mean for the average developer working on an upgrade from 6.0.4 to 6.1? Not very much. The database upgrade scripts included in the 6.1 distribution take care of all the necessary schema changes.

 

  • Populating the settings tables with the appropriate values is critical to getting your 6.1 install up and running. For more information, see the Elastic Path 6.1 Upgrade Guide at http://docs.elasticpath.com/display/EP61UPGRADE.
  • Increasing the size of a few columns should have minimal impact.
  • Removing the "required" flag on some fields in the STORE table is essential to the Store States feature, but should have no impact on existing systems.
  • TSTORE.STORE_STATE - 6.0.x only had a single store state: OPEN. The OPEN state in 6.1 is represented by an integer value of 200 in this field.
  • TCMUSER.STATUS replaces the ENABLED column but is backward compatible with its values. The value of the STATUS column should be the same as the value of the ENABLED column for every record during upgrade.
  • The new indexes can be added without any problems.
  • Removing the unused TPROMOCODE table should have no impact, but if you have customizations that are using this table, then it can also be left in with no impact.
  • Creating the new TSTORESUPPORTEDCURRENCY and TSTORESUPPORTEDLOCALE tables for Stores, and creating records corresponding to the column values in the corresponding columns that were removed from TSTORE should be fairly easy.

 

Upgrading an Elastic Path database-from 6.0.x to 6.1 is relatively painless. Only one integer field's data must be translated to a new field and removed (TCmUser.Enabled to TCmUser.Status), and only one new integer field (TStore.store_state) must added and populated with data (but every row has the same value for an upgrade). Other schema changes should have no significant impact on a stock EP6.0.x database.

0 Comments 0 References Permalink