How A/B Testing Can Help Drive Ecommerce Performance | Salsify

The pressure to be present on the ever-increasing number of digital channels has transformed commerce in recent years, but some brands are undertaking this journey somewhat blindly. While best practices for product pages offer valuable insights for establishing a foundational strategy, they can only take brands so far.

Brands must also take steps to understand what drives discovery and performance for their unique offerings. A/B testing, which splits users evenly between two versions of a webpage to give teams insights into how well certain variables perform, allows brands to gain a deeper understanding of what drives both successes and failures on the digital shelf.

Limitations of A/B Testing on the Digital Shelf

While A/B testing is a common practice for most digital teams, allowing them to gain insights about critical metrics like traffic and conversion across owned channels, the digital shelf cannot offer the same level of detail.

“There are huge confounding factors that make getting real statistical significance difficult,” said Rob Gonzalez, co-founder and CMO of Salsify. Highlighting product titles as an example, Gonzalez stressed that there are so many moving parts on the digital shelf that it would be difficult to gauge whether any changes impacted conversion for a specific product.

The product could have gone out of stock for a period of time — or a competitor’s product could have gone out of stock, driving demand for alternative offerings. The Amazon Marketing Services (AMS) team could have run out of budget. A competitor could have been running an Instagram campaign to drive traffic to their product page.

These factors are often out of the control of brands, making it challenging to draw causation on whether a product title change made a difference.

How to Implement Ecommerce A/B Testing

Despite these limitations, there are ways to gain insights through A/B testing that can help boost performance across key business metrics.

Step 1: Identify Content for Ecommerce A/B Testing

Brands must complete A/B testing on a broad scale. Through examining a vast selection of products across a single product category or groups of product categories, brands can identify patterns. These patterns can then be used to glean insights that allow teams to optimize product listings.

Gonzalez stressed that Salsify examines hundreds of thousands — and sometimes millions — of products to identify patterns. From these insights, Salsify develops best practices that can help brands optimize performance on the digital shelf, including the fact that more product bullets, photos, and reviews can help product pages convert at a higher rate.

ShurTech, a subsidiary of ShurTape Technologies, an industry-leading producer of pressure-sensitive masking, duct, packaging, and specialty tape products, used A/B testing to find patterns within its product offerings. After optimizing 70 of its products, ShurTech saw a 15% increase in sales rank across its entire range of products.

Through examining large portions of products, brands can find the information needed to optimize listings and boost online presence and revenue.

Step 2: Get Specific About Measurement Focus Areas

As with all marketing metrics, brands must first understand precisely what it is they want to achieve through ecommerce A/B testing. The more specific brands are, the more useful the data will be.

Avoid combining multiple factors into one test. For example, analyzing image type, image number, and hero image placement at the same time will muddle the data. This push for too many metrics is one of the biggest mistakes brands make when A/B testing.

Brands must consider business objectives and test the metrics that best align with those goals. Specificity makes testing more simple — as it avoids excessive data that takes substantial time and effort to analyze — and helps brands identify the key insights that can help future product page optimization.

Step 3: Start Small Before Scaling

Being specific will also help brands narrow down their choices — but getting too niche will not bear robust results. Brands might not be able to perform ecommerce A/B testing on just one product, but doing so across a range of products is possible if brands have cohorts that are small enough.

Testing on the digital shelf is not yet that sophisticated. Brands won’t get data fast enough for large-scale tests, nor will they be able to set up experiments fast enough for a large number of products. 

The best thing to do for brands is to start with small sets of products and specific metrics they want to study. From there, brands can build out testing and knowledge bases.

Optimize Future Commerce Strategy

While A/B testing on the digital shelf has its limitations, it can provide brands with valuable information that can guide future commerce strategy. Through developing a better understanding of product categories, brands can take steps forward to evolve their digital presence.