NB: A familiarity with Sitecore marketing features is assumed.
Upgrading to Sitecore 8 is an exciting moment, and most organizations are eager to start using the powerful marketing features in this new version. However, sometimes the solution requires some tweaking to ensure everything is working correctly. This is most commonly due to an older solution that was not built in context of the marketing features.
When we validate a solution to ensure there are no obstacles to using Sitecore 8, we call this an “XP Readiness Check”. From a QA perspective, we split the testing into two categories so testing time can be allocated as efficiently as possible:
- The Kernel, which comprises the XP marketing features we recommend most commonly, also known as quick wins or the more accessible features
- The Case-By-Case, which comprise the XP marketing features that require a case-by-case analysis, or are more pertinent to advanced use
It goes without saying that there are a few prerequisites to ensuring your upgraded solution is working properly, including proper xDB configuration and upgrade dragons are slain.
Once these are verified, the tables below can be used to guide this readiness Quality Assurance check. It is not meant to be exhaustive, but rather should indicate whether marketing feature use can proceed with relatively few obstacles, or whether larger remediation is required.
The Kernel – prioritize evaluation of these
||Things to look for
||All Experience Editor modes load properly
A subset of template types and/or pages are tested to load in Experience Editor, Preview Mode and Explore mode with no JS errors or conflicts.
- A small analysis can generate a list of top 10 high traffic pages vs. optimization candidates (homepages, conversion pages, landing pages) vs. most used templates
- You can follow these tips as well
The personalization wizard loads correctly
Personalization can be applied in desired areas
Templates are component-driven
On an optimization candidate (homepage and/or landing page), the personalization wizard works end-to-end and component architecture is available as expected in the right and left rails
GeoIP service and firewall rules are configured correctly
You have switched the service from MaxMind to Sitecore
|A personalization rule is tested using a GeoIP rule
- Quick test: The rule works in Explore mode
- Robust test: The rule is published and tested in a live environment
||Expected traffic is appearing
No collection errors appearing in logs
Raw data appears in Mongo (if running on-premise)
Additional tips here
Verify that traffic is appearing as expected in Experience Analytics.
Test variation preview is configured and working in the wizard
Workflows are configured for testing
Experience Optimization reports, such as Active Tests, are displaying correct results
|There are three routes to configure Content Testing in v8:
- Content testing (triggered from workflow)
- Component testing (triggered from workflow)
- Page testing (triggered from wizard)
Our most commonly recommended tactics involve #1 and #2, therefore I recommend both are tested. This will require some configuration in the workflow and instructions can be found in the links above.
Testing should include:
- Successful creation of a test variant and publication of test
- Verification that different users are seeing different variations
- Verification that the Active Tests report (in Experience Optimization) and the Optimization report (from the Experience Editor ribbon) are displaying expected results
|Web Forms for Marketers (if applicable)
Typically upon upgrade, tweaks are necessary if custom fields, custom save actions, .config patches and heavily styled forms are already in place.
These patches and customizations must be made compatible with the latest version.
We recommend testing an end-to-end scenario with a simple lead gen form example:
- Add a new form to a new page where Placeholder Settings are already configured
- Configure fields and a simple submission action, depending on your context and connectivity to CRM
- Publish form
- Test validators
- Test submit actions
Features to evaluate case-by-case
In this table, we outline our recommendations for testing these Sitecore marketing features which require contextual consideration.
|| Testing recommendation
The path analyzer provides a sophisticated view of traffic flows and optimization opportunities. It should be included in the test plan if the client intends to use it for regular website performance analysis.
- Quick test: Path Analyzer loads without error and displays traffic data
- Robust test: Engagement value tagging is applied, and Path Analyzer displays colour-coding as expected
||Engagement plans provide marketing automation and lead nurturing functionality. They should be included in the test plan if the client’s marketing roadmap includes automated nurturing after a form submission or campaign click-through.
We recommend testing a quick end-to-end scenario. This will require some initial setup.
- Configure a quick 2-step engagement plan that sends an email after a short elapsed period
- Configure a campaign that enrolls a user into an engagement plan
- Load a page in the site with the campaign query parameter appended (be careful to do this in a new browser session)
- Confirm in Monitor / Supervise mode that the user is moving through the states as expected
- Confirm receipt of email after elapsed time
Usage of the Experience Profile is typically highly specific to an organizational context, especially in terms of Customer System of Record and integration with other systems such as CRMs.
One of the quickest ways to validate basic functionality is to create a Sitecore Contact via submission of Web Forms for Marketers, and check the Experience Profile report for record creation. If you converted any goals during this session, they should also appear in the profile record.
NB: Rebuilding the reporting database post-upgrade can resolve some Experience Profile issues:
- Back up the Data\indexes\sitecore_analytics_index folder.
- Delete all content of the Data\indexes\sitecore_analytics_index folder.
- Perform the rebuild of the reporting database.