AB Testing


August 08, 2014

Offering free shipping on all orders over $49, Increases eCommerce gross Profit after shipping by 28%.

 
 

We ran four different versions (an A, B, C, D test) on zumba.com (US only) to analyze whether adding “free shipping” would impact Conversion Rate (CR) and Average Order Value (AOV). Site (zumba.com) visitors were equally divided into 4 groups and shown different pricing offers/thresholds.The breakdown of the 4 offers were: free shipping over $49, free shipping over $99, free shipping with no minimum, and no promotion offer. The fourth offer (no promotion) was to be used as a control group for comparison.

These promotions were not advertised, allowing us to gauge the natural organic lift. The results revealed that revenue per visit increased by 16.83% with a 98% confidence level, when new and returning visitors were offered free shipping over $49. In addition to this, the Gross Profit after shipping
rose by 28%.

It is important to note, however, that offering free shipping with no minimum has a greater CR (by 22.38% over the control group). However, the AOV went down by -11,88%. In addition, the free shipping over $99 increased AOV by 4.83%. However, CR went down by -.0.97%.

Free shipping has become the expectation in the marketplace from consumers. The data shows that offering free shipping over $49 will help elevate the Zumba shopping experience and make us more competitive in today’s marketplace. Additionally, free shipping will be added to search copy, display ads, and marketing e-mails to help drive more traffic to our site, thus increasing sales and overall brand awareness.

 
 

ab-freeship
 
 
 
 

Changing the URL to drive traffic to the “Become an Instructor” page for people looking for trainings can generate 14% more registrations.

 

We tested four different experiences (an A, B, C, D test) on zumba.com (US and Desktop only) to analyze whether the placement and URL impacts Conversion Rate for training registrations. Site (zumba.com) visitors were equally divided into 4 groups and shown different placement and URLs for “Instructor Trainings”. The breakdown of the 4 experiences were:

 

  • A. No change (control group)
    B. Same placement with link to the “Become an Instructor page” (http://www.zumba.com/en-US/trainings)
    C. Move the placement of “Instructor Trainings” so that its part of the main navigation (linking to the B1 page) (http://www.zumba.com/en-US/trainings/basic1)
    D. Move the placement of “Instructor Trainings” so that its part of the main navigation with link to the Become an instructor page (http://www.zumba.com/en-US/trainings)
    No other change was made to the site and training registrations were not advertised, allowing us to gauge the natural organic lift.

 

The test started with very clear differences between the groups, with one group emerging as a clear winner early on. As it continued to run, we saw changes in behavior between new and returning visitors. Moreover, we have proven that the “Become an Instructor” training that links to the “Become and Instructor page” (experiences B and D) performs better than the B1 page (experience C). In order to get faster results, and understand the best placement for the link, we removed the lowest-performing experience C (Move “Instructor Trainings” into the main navigation and linking to the B1 page).

With a strong confidence level of 90%, the results revealed that training registrations in the US increased by 14%, when new and returning visitors looking for trainings were directed to the “Become an Instructor” page from the main navigation. This is test group D (Move “Instructor Trainings” into the main navigation and linking to the Become an Instructor page).

I recommend adding “Instructor Trainings” to the main navigation to help increase training registration. Additionally, it would help elevate the brand’s credibility by showing the education programs.

 
 
ab-trainings