Are Your P-values Killing your AB Testing Efforts?

If you’re running AB tests, you’re surely running into concepts like “95% confidence intervals” and the “Null hypothesis”. And you’re already using “p-values”, too, though you may not know it.

Unfortunately there’s a great chance you’re using them wrong, and it’s having an impact on your AB testing program?

Here is a business person’s overview to help you avoid common problems and misunderstandings about p-values.

Read more

Making Millions From Losing Tests (Part 1)

Over the course of this series of columns, we’re going to develop a spreadsheet that you can use to model your own company’s efforts towards testing, each month adding more and more features to the underlying model.

The bigger the company, it seems, the more disconnect there can be between the talented testing and optimization team and senior management. And that, as you might guess, ends up undermining large portions of the revenue and efficiency improvements that come from continuous testing.

Read more