Many online retailers operate across multiple markets. This use case demonstrates how to set up an experiment using conditions to manage traffic from specific markets.

Our default checkout configuration offers three delivery options. We want to test how introducing five delivery options impacts cart conversion, limiting the test variant exclusively to Swedish customers.

Learn how to:

1. Set up the Checkout configuration
2. Configure the experiment
3. Update your Connection to use an Experiment
4. View the test results

Set up the Checkout configuration

First, you need to configure the Checkout configuration that you want to compare to your default Checkout. The easiest way to do this is by duplicating your default configuration and changing the part you want to test.  

1. In nShift Portal go to Checkout > Configurations and locate your default checkout.
   
   
2. Click on the three dots menu and choose Duplicate.
   
   
   
   
3. Update the Name of the configuration to differentiate it from the original configuration. The name is only for internal use and will be displayed in the Checkout’s configuration interface.
   
   
4. Select the member account where the Checkout configuration should be saved. 
   
   
5. Select a template and a sender. All other settings will be prefilled with the data from the original configuration. 
   
   
6. Go to Delivery options and add the additional delivery options you want to include in the test variant. 
   
   
7. Click Save. 

Note: If you are setting up a Checkout configuration from scratch, you can find instructions in this article: Create an nShift Checkout configuration

Configure the experiment

We are now ready to configure the experiment. We want to test the conversion rate for Swedish customers only so we will split the traffic from Sweden 50-50 between the default checkout configuration and the test variant. All traffic from other countries will be sent to the default configuration.

1. Go to Experiments in the top menu and click Create new experiment.
   
   
2. Fill in Name and choose the Member account from the drop-down list (should be the same as used for your checkout configurations).
   
   
3. Add a description to remember the purpose of the experiment.
   
   
4. Enter the Start and Stop date of the experiment. The experiment will only run during this period. When the current date is before or after the experiment period, the configuration or group defined in the connection will be used.
   
   
5. Choose to_country (receiver country of the order) from the Parameters drop-down list. 
   
   
6. Click Add new segmentation and activate the Add condition toggle. In this example, it is specified that if the customer is from Sweden, 50% of the traffic should be directed to the test variant (Experiment Sweden) and 50% should be sent to the original configuration (Checkout EU).
   
   
   
   
7. Click Continue.
   
   
8. Click Add new segmentation and activate the Add condition toggle. 
   
   
9. Fill in a condition - here it is specified that if the customer is not from Sweden, they should be directed to the default checkout configuration (Checkout EU). 
   
   
   
   
10. Click Continue and Save. The experiment is now ready and will run for the planned time. 
    
    
11. If you are using our API, remember to update your connection to point to the experiment (read more here).

Image showing the entire experiment setup:

The Checkout widget examples

Configuration A: Three delivery options.

Configuration B: Five delivery options.

Update your Connection to use an Experiment

If you are using the Checkout API, your requests contain a Connection ID to identify your Checkout configuration. To use an Experiment in your current integration, you simply point your connection to an Experiment. This is easily done in the user interface and requires no code changes. 

1. Go to Connections in the top menu.
   
   
2. Click the three-dots menu next to the Connection used by your integration and choose Edit.
   
   
3. Select the Experiment to be used from the Experiment drop-down list. 
   
   
   
   
4. Click Save. The Connection ID will now point to the Experiment instead.

View the test results

When the experiment is completed, you can download a metrics file to analyze the results. 

1. Go to Experiments in the top menu.
   
   
2. Click the three-dot menu next to the experiment for which you want the results and choose Download Results.
   
   
   
   
3. Click the Export button in the pop-up. The dates for the experiment are prefilled but you can adjust them if needed. 
   
   
   
   
4. A CSV file will be generated soon with the data collected during the test period.

You can also see key metrics under Segmentations > Results on the experiment configuration page.

Metrics data in the report

Below is an overview of the metrics included in the CSV file. Each line in the file represents a checkout session and its data.

• Member Account ID
• Request ID
• Connection ID
• Group ID
• Checkout ID
• Checkout Version
• Receiver Postal Code
• Receiver Country Code
• Language Code
• Currency Code
• Option ID
• Option Name
• Option Index
• Segmentation ID
• Source System
• Carrier ID
• Carrier Product ID
• Created
• Selected
• Shipment Count
• Total Request Price
• Total Checkout Price
• Total Addons Price
• Total Full Price