All entries by this author

Are Your P-values Killing your AB Testing Efforts?

Sep 4th, 2014 | By
Are Your P-values Killing your AB Testing Efforts?

If you’re running AB tests, you’re surely running into concepts like “95% confidence intervals” and the “Null hypothesis”. And you’re already using “p-values”, too, though you may not know it.

Unfortunately there’s a great chance you’re using them wrong, and it’s having an impact on your AB testing program?

Here is a business person’s overview to help you avoid common problems and misunderstandings about p-values.



Thinking Inside The Boxplot

Oct 1st, 2013 | By
Thinking Inside The Boxplot

In a previous post describing a simple approach to de-seasonalizing your data, I covered how marketers can examine, at a rough level, the impact of
[continue reading…]



Tis The Seasonality Of Your Metrics

Jul 11th, 2013 | By
Tis The Seasonality Of Your Metrics

I want to show you a technique to fix another common time-series problem: seasonality. The article will illustrate a fast and simple way to de-seasonalize your data.



Keep An Infinite Memory for your Metrics

May 15th, 2013 | By

The Exponential Moving Average (“EMA”) is a very useful tool in your metrics arsenal because in the EMA the memory of past metrics values is never forgotten



Setting Up Your Negative Metrics

Apr 17th, 2013 | By

This is a continuation of  my exploration of negative metrics thread which covers how to think about your negative metrics and how to set them up
[continue reading…]



Thinking About Your Negative Metrics

Apr 16th, 2013 | By

“Negative” metrics — You might prefer the term “Deoptimization Metrics”. Either way they can be just as important  to your continuous optimization efforts as your positive ones. The purpose of a negative metric  is to isolate for you the deleterious effect you may be having on your positive metric. A negative metric is not necessarily
[continue reading…]



I Before E, Except After C

Mar 19th, 2013 | By
I Before E, Except After C

“I Before E, Except After C” — do you remember that from 4th grade spelling? Well, it works just as well when you’re concentrating on
[continue reading…]



Treating Your Conversion Symptoms But Not The Disease

Jan 21st, 2013 | By
Treating Your Conversion Symptoms But Not The Disease

Many organizations get fixated on metrics data, without thinking of where it comes from. But what if your metrics indicate a problem which originates somewhere else?



Polls and Testing: When Is Close No Cigar?

Nov 20th, 2012 | By
Polls and Testing: When Is Close No Cigar?

Now that the 2012 U.S. Presidential elections are over, there’s a bit of a buzz around why some folks thought the election would be close and others who, using some solid statistical techniques, predicted a definitive (and it turns out, accurate) outcome. I want to try to explain the latter, especially for marketers. Why? Because
[continue reading…]



I, Deceiver

Oct 2nd, 2012 | By
I, Deceiver

This is a follow up to my previous article “Oh, What A Tangled Web We Weave” which discusses 8 common ways we practice self-deception on
[continue reading…]