Are customer surveys worth the paper they are written on?

Share this content

In our daily business selling printer ink on the internet, we are told by the marketing experts that you absolutely must test any changes you make to your store and that you absolutely must gather customer feedback and implement whatever trends you see emerging. So we slavishly adhere to these principles.  

Testing has become a part of our culture, but it didn’t start out that way.  I remember back in 2009 when we first began using Google Optimiser we were making subtle changes to our printer ink website and tracking them with Optimiser. We were seeing a 40 - 50% increase in conversions from the changes in the first couple of days, and I had dreams of retiring to a beach in the Bahamas. However after about a month of running the test, when the numbers became statistically significant we actually saw very little difference and I was brought back to reality!

This is my point in this blog; you need to understand what the results of testing or customer feedback are actually showing you before you go all out and implement changes. In the case of Optimiser we now know that subtle changes are too small to be recorded accurately; you need to be testing big changes. Also you can’t take the first few days’ results as Gospel, you have to give it a month or more before drawing any conclusions.

Likewise with customer feedback. We have always had some means of garnering customers’ views on the website, and my philosophy has been to use the positive comments to develop and push our unique selling points (USPs), and to use any negativity to address problem areas as a matter of urgency. However, when Walmart got itself into a customer-driven change mentality it cost them an alleged $1.85 billion - not an insignificant amount I’m sure you will agree.  

According to a story on Daily Artifacts, back in 2008 Walmart conducted a customer survey which found that the majority of customers wanted a “less cluttered” store, so it rolled out a strategy which revised its decades-old strategy of low price and wide selection.

  • 15% of the inventory removed from the stores. Some suppliers reported losing 30% of their stock in Walmart stores due to the revamp
  • Pallets of items like juice boxes or sweatshirts stacked in the centres of aisles were removed
  • Merchandise on “end caps” displays at the ends of aisles was slimmed down
  • Shelves were shortened as well
The revamp not only removed items but cost "millions of dollars" per store in refurbishment costs. The exercise saw an immediate loss in revenue and decline in same-store sales data (see the chart by clicking on the image at the top of this post).

In hindsight Walmart relied on what customers said in a survey, rather than what it already knew from customer behaviour in store. It took this research as the truth and rolled out a huge programme of change that affected the value of the store group to the tune of an estimated 1.85 billion dollars. Some argue that this happened at the same time as the global recession was hitting people’s spending power, however Walmart with its low prices and value model, should have been in a prime position to benefit when budgets get tight.

So, the rather expensive lesson that we can all learn from this is:
  • Test, but allow time for statistically significant differences to be apparent
  • Gather customer feedback, but test changes before rolling them out store-wide
Always listen to your customers, but sometimes, just occasionally, they might be wrong!


Please login or register to join the discussion.

By bizdreams
22nd Aug 2011 23:05

I can't believe Walmart did such a thing, ofcourse a customer would say that. we all say we want less cluttered homes but can't throw away any of our possessions.

Thanks (0)