Delivery costs can be critical for cart conversion rates. In this use case, we will compare two delivery fee options for orders below the free shipping threshold:

• Configuration A: A delivery fee of 49 NOK
• Configuration B: A reduced delivery fee of 39 NOK
• The free shipping threshold is 300 NOK in both configurations.

Learn how to:

1. Set up the Checkout configuration
2. Configure the experiment
3. Update your Connection to use an Experiment
4. View the test results

Set up the Checkout configuration

First, you need to configure the Checkout configuration that you want to compare to your default Checkout. The easiest way to do this is by duplicating your default configuration and making changes to the part you want to test. 

1. In nShift Portal go to Checkout > Configurations and locate your default checkout.
   
   
2. Click on the three dots menu and choose Duplicate.
   
   
   
   
3. Update the Name of the configuration to differentiate it from the original configuration. The name is only for internal use and will be displayed in the Checkout’s configuration interface.
   
   
4. Select the member account where the Checkout configuration should be saved. 
   
   
5. Select a template and a sender. All other settings will be prefilled with the data from the original configuration. 
   
   
6. Our default Checkout has a delivery fee of 49 NOK for orders below 300 NOK. This one should have a delivery fee of 39 NOK for orders below 300 NOK. Go to Delivery options and change the price of the delivery options to 39 NOK.
   
   
7. Click Save. 

Note: If you are setting up a Checkout configuration from scratch, you can find instructions in this article: Create an nShift Checkout configuration

Configure the experiment

We are now ready to configure the experiment. We want to take all orders below the free shipping threshold and send 50% of the traffic to our default checkout and 50% to the new configuration we created for the experiment. All orders that meet the free shipping threshold should be sent to the default configuration.

1. Go to Experiments in the top menu and click Create new experiment.
   
   
2. Fill in Name and choose the Member account from the drop-down list (should be the same as used for your checkout configurations).
   
   
3. Add a description to remember the purpose of the experiment.
   
   
4. Enter the Start and Stop date of the experiment. The experiment will only run during this period. When the current date is before or after the experiment period, the configuration or group defined in the connection will be used.
   
   
5. Choose cart_price from the Parameters drop-down list.
   
   
6. Click Add new segmentation and activate the Add condition toggle. In this example, it is specified that if the cart price is less than 300 which is our free shipping threshold then 50% of traffic should be sent to the "Delivery Costs Test" configuration (the one we created above) and the other 50% should be sent to the "Checkout for Norway" (our default configuration).
   
   
   
   
7. Click Continue.
   
   
8. Click Add new segmentation again and activate the Add condition toggle.
   
   
9. Specify which configuration to use when the free shipping threshold is met. Choose cart_price is greater than or equal to 300 and send 100% of traffic to the default configuration. 
   
   
   
   
10. Click Continue and Save. The experiment is now ready and will run for the planned time. 
    
    
11. If you are using our API remember to update your connection to point to the experiment (read more here).

Image showing the entire experiment setup:

The Checkout widget examples

Configuration A: Default prices of 49 NOK.

Configuration B: Lower prices of 39 NOK.

Update your Connection to use an Experiment

If you are using the Checkout API, your requests contain a Connection ID to identify your Checkout configuration. To use an Experiment in your current integration, you simply point your connection to an Experiment. This is easily done in the user interface and requires no code changes. 

1. Go to Connections in the top menu.
   
   
2. Click the three-dots menu next to the Connection used by your integration and choose Edit.
   
   
3. Select the Experiment to be used from the Experiment drop-down list. 
   
   
   
   
4. Click Save. The Connection ID will now point to the Experiment instead.

View the test results

When the experiment is completed, you can download a metrics file to analyze the results. 

1. Go to Experiments in the top menu.
   
   
2. Click the three-dot menu next to the experiment for which you want the results and choose Download Results.
   
   
   
   
3. Click the Export button in the pop-up. The dates for the experiment are prefilled but you can adjust them if needed. 
   
   
   
   
4. A CSV file will be generated soon with the data collected during the time period.

You can also see key metrics under Segmentations > Results on the experiment configuration page.

Metrics data in the report

Below is an overview of the metrics included in the CSV file. Each line in the file represents a checkout session and its data.

• Member Account ID
• Request ID
• Connection ID
• Group ID
• Checkout ID
• Checkout Version
• Receiver Postal Code
• Receiver Country Code
• Language Code
• Currency Code
• Option ID
• Option Name
• Option Index
• Segmentation ID
• Source System
• Carrier ID
• Carrier Product ID
• Created
• Selected
• Shipment Count
• Total Request Price
• Total Checkout Price
• Total Addons Price
• Total Full Price