Are Your P-values Killing your AB Testing Efforts?

Headline

Are Your P-values Killing your AB Testing Efforts?

If you’re running AB tests, you’re surely running into concepts like “95% confidence intervals” and the “Null hypothesis”. And you’re already using “p-values”, too, though you may not know it.

Unfortunately there’s a great chance you’re using them wrong, and it’s having an impact on your AB testing program?

Here is a business person’s overview to help you avoid common problems and misunderstandings about p-values.

[continue reading...]

Featured

Thinking Inside The Boxplot Thinking Inside The Boxplot

In a previous post describing a simple approach to de-seasonalizing your data, I covered how marketers can examine, at a rough level, the impact of
[continue reading…]

Tis The Seasonality Of Your Metrics Tis The Seasonality Of Your Metrics

I want to show you a technique to fix another common time-series problem: seasonality. The article will illustrate a fast and simple way to de-seasonalize your data.

Keep An Infinite Memory for your Metrics

The Exponential Moving Average (“EMA”) is a very useful tool in your metrics arsenal because in the EMA the memory of past metrics values is never forgotten

Metric-ocracy: Less Data, More Insight

The start of a new year is a popular time for resolutions. Of course, even among marketing cognoscenti new year’s resolutions rarely involve analytics and key performance indicators (KPIs) — so I’d like to take this opportunity to outline a plan you can follow to slim down and shape up in 2012, metrics-wise.

Treating Your Conversion Symptoms But Not The Disease Treating Your Conversion Symptoms But Not The Disease

Many organizations get fixated on metrics data, without thinking of where it comes from. But what if your metrics indicate a problem which originates somewhere else?

Are Your P-values Killing your AB Testing Efforts? Are Your P-values Killing your AB Testing Efforts?

If you’re running AB tests, you’re surely running into concepts like “95% confidence intervals” and the “Null hypothesis”. And you’re already using “p-values”, too, though you may not know it.

Unfortunately there’s a great chance you’re using them wrong, and it’s having an impact on your AB testing program?

Here is a business person’s overview to help you avoid common problems and misunderstandings about p-values.

Dear Confused By Personas Dear Confused By Personas

A student at the University of British Columbia Web Analytics course reached out to us via Twitter to ask some questions about creating personas, specifically personas based on persuasion and the information is important enough that we thought we’d share our response: “Are you saying that we shouldn’t bother with creating multiple personas with granular
[continue reading…]

The Value of Nothing The Value of Nothing

When is nothing worth something? Turns out, just about all the time. Whenever you look at your metrics consider that a metric is, in virtually
[continue reading…]

Treating Your Conversion Symptoms But Not The Disease Treating Your Conversion Symptoms But Not The Disease

Many organizations get fixated on metrics data, without thinking of where it comes from. But what if your metrics indicate a problem which originates somewhere else?

PayPal Should Go Undercover

PayPal recently announced a streamlining of its payment flow process that doesn’t require a PayPal account to use. In other words, you can “check out” via PayPal, reap the security benefits of the merchant store not knowing your financial details, and pay for your item without having created any long-term relationship with PayPal (although they
[continue reading…]

Polls and Testing: When Is Close No Cigar? Polls and Testing: When Is Close No Cigar?

Now that the 2012 U.S. Presidential elections are over, there’s a bit of a buzz around why some folks thought the election would be close and others who, using some solid statistical techniques, predicted a definitive (and it turns out, accurate) outcome. I want to try to explain the latter, especially for marketers. Why? Because
[continue reading…]