Headline

Are Your P-values Killing your AB Testing Efforts?

Sep 4th, 2014 | By
Are Your P-values Killing your AB Testing Efforts?

If you’re running AB tests, you’re surely running into concepts like “95% confidence intervals” and the “Null hypothesis”. And you’re already using “p-values”, too, though you may not know it.

Unfortunately there’s a great chance you’re using them wrong, and it’s having an impact on your AB testing program?

Here is a business person’s overview to help you avoid common problems and misunderstandings about p-values.



Tis The Seasonality Of Your Metrics

Jul 11th, 2013 | By
Tis The Seasonality Of Your Metrics

I want to show you a technique to fix another common time-series problem: seasonality. The article will illustrate a fast and simple way to de-seasonalize your data.



Keep An Infinite Memory for your Metrics

May 15th, 2013 | By

The Exponential Moving Average (“EMA”) is a very useful tool in your metrics arsenal because in the EMA the memory of past metrics values is never forgotten



Thinking About Your Negative Metrics

Apr 16th, 2013 | By

“Negative” metrics — You might prefer the term “Deoptimization Metrics”. Either way they can be just as important  to your continuous optimization efforts as your positive ones. The purpose of a negative metric  is to isolate for you the deleterious effect you may be having on your positive metric. A negative metric is not necessarily
[continue reading…]



Treating Your Conversion Symptoms But Not The Disease

Jan 21st, 2013 | By
Treating Your Conversion Symptoms But Not The Disease

Many organizations get fixated on metrics data, without thinking of where it comes from. But what if your metrics indicate a problem which originates somewhere else?



Polls and Testing: When Is Close No Cigar?

Nov 20th, 2012 | By
Polls and Testing: When Is Close No Cigar?

Now that the 2012 U.S. Presidential elections are over, there’s a bit of a buzz around why some folks thought the election would be close and others who, using some solid statistical techniques, predicted a definitive (and it turns out, accurate) outcome. I want to try to explain the latter, especially for marketers. Why? Because
[continue reading…]



The Value of Nothing

Aug 30th, 2012 | By
The Value of Nothing

When is nothing worth something? Turns out, just about all the time. Whenever you look at your metrics consider that a metric is, in virtually
[continue reading…]



Would You Tip You?

Jul 9th, 2012 | By
Would You Tip You?

In many ways, being effective online means paying attention to the things that matter. That recipe includes conversion, and analytics, and metrics — important ingredients,
[continue reading…]



Making Millions From Losing Tests (Part 1)

Feb 17th, 2012 | By
Making Millions From Losing Tests (Part 1)

Over the course of this series of columns, we’re going to develop a spreadsheet that you can use to model your own company’s efforts towards testing, each month adding more and more features to the underlying model.

The bigger the company, it seems, the more disconnect there can be between the talented testing and optimization team and senior management. And that, as you might guess, ends up undermining large portions of the revenue and efficiency improvements that come from continuous testing.



Why Test? The Purpose of Testing

Jan 17th, 2012 | By
Why Test? The Purpose of Testing

Why Test? The purpose of testing is not to find out what works, but rather to find out what does not work. I often encounter clients who have what they consider a large percentage of “failed” tests. Yet these tests reveal a rather large amount of information and insight towards future testing. In fact, when a test
[continue reading…]