Book – You Should Test That!

bookThis book exceeded my expectations!  It is a wonderful resource with many examples to refer and an elegant methodology that should be easy to follow.

The book is not limited to the theory of marketing testing; it includes a wonderful overview of the relevant aspects of the entire marketing discipline.

A great quote from the book attributed to scientist Peter Medawar

The intensity of conviction that the hypothesis is true has not bearing over where it is true or not.

My notes (not a book summary; a few quick points that I think were explained exceptionally well or were new to me):

  • CRO (conversion rate optimization) and SEO are not mutually exclusive.  “You will achieve better search engine rankings with pages that focus on a single topic or product.  This will improve you conversion rate, too.”
  • bounce-rateReasons not to optimize for bounce rate
    • The page might be attracting unrelated visitors (organic ranking for unrelated items)
    • The page has all needed information and does not require browsing the site
    • Bounce rate can be lowered without any business benefit:
      • changing one page to a microsite won’t increase purchases, but will reduce bounce rate
      • moving key information (pricing) to another page won’t increase sales either
      • adding popular links can reduce bounce rate and also destruct/reduce conversions
      • replacing productive offers with free offers may reduce revenue
  • templatePrioritizing tests: though the home page might be perceived as the most reasonable place to test (traffic!) it might be politically difficult.  Even if the home page is the most popular page of the site, the aggregate of all pages that use the same template can be higher.  This template could be a primary candidate for testing.  Also test most popular landing pages
  • A recommendation to use 5 seconds test (very good idea!  I see it periodically)
  • 3 questions from 4Q (love 4Q!)
    • What is the purpose of your visit to our website today?
    • Were you able to complete your task today?
    • If you were not able to complete your task today, why not?
  • Test pages – 20,000 – 30,000 monthly unique visitors.  Statistically significant result can be achieved within a few weeks.  If traffic is less, the test would take longer.

If your value proposition is strong and the communication of it is weak, you can have large conversion-rate lift from improvements to your pages.  In other words, if people want what you offer, but they can’t figure out what you are saying, simply saying it more clearly will motivate more of them to buy!

On the other hand, if your value proposition is weak, or weaker than that of competitors, you’ll have a difficult time improving your conversion rates, no matter how well designed your website and landing pages are.

  • Conversion rate elasticity – ability to improve your conversion rate
  • LIFT model
    • liftRelevance
    • Clarity
    • Anxiety
    • Distraction
    • Urgency


  • Use anchoring for prices presentation – high price on left (for left-to-right reading locations).  Unbounce has a good pricing example.


Wireframes should be in black and white to avoid preoccupation with design elements and are intentionally not pixel-precise.  At the wireframing stage, you should be thinking about eye-flow, the content hierarchy, and the visitor decision-making process.

  • In general, reducing number of columns improves conversion rate – particularly important for forms.  One-column form is typically performs better.
  • Use graphic elements (direction) – even in tweets.  In a tweet an arrow –> can point to the registration URL, etc.
  • If images on a web page use captions, they are most likely to be read.  They can be called “action captions – and should include an offer statement.
  • softwareImages on the page
    • should support value proposition
    • should be directional leading to the call to action
    • should be high quality for retailers
    • consistent alignment improves clarity and reduces visual processing
    • images also important for companies with intangible products or services.  The challenge is to make a product or service more tangible.  (Software vendors show software boxes, or screenshot of product interface)
    • Do not use “clever” image concepts – they reduce clarity
  • Testing headlines – easiest way to find most powerful motivational trigger message
  • Form option – moving optional form fields to the thank you
  • Adding security items to the shopping cart needs to be tested.  One company placed McAfee Secure symbol in the site-wide shopping cart area and saw reduction of e-commerce sales by nearly 2%

Especially above the fold, your paragraphs should be no more than three lines high.

…sales reps had a 100x greater chance of successfully contacting a lead if the first call was within 5 minutes after the lead form was filled out rather than 30 minutes.  And if the sales rep did not call for 5 hours, their success rate decreased by 3000x

  • If manual entry of phone numbers would be required in order to track revenue, don’t track it.  The inaccuracy of manual phone number entry is much greater than that of estimating phone lead value.  If you can not track phone call revenue, track phone calls over a certain call length as conversions.

The book has a wonderful companion site with downloadable case studies and discussions associated for each chapter.


Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s