# Napkin Folding — A/B testing

## Bayesian Methods for Hackers release!

Posted by **Cameron Davidson-Pilon** at

Finally, after a few years writing and debugging, I'm proud to announce that the print copy of Bayesian Methods for Hackers is released! It has update content, including a brand new chapter on A/B testing, compared to the online version. You can purchase it on Amazon today!

## The Class Imbalance Problem in A/B Testing

Posted by **Cameron Davidson-Pilon** at

Introduction If you have been following this blog, you'll know that I employ Bayesian A/B testing for conversion tests (and see this screencast to see how it works). One of the strongest reasons for this is the interpretability of the analogous "p-value", which I call Confidence, defined as the probability the conversion rate of A is greater than B, $$\text{confidence} = P( C_A > C_B ) $$ Really, this is what the experimenter wants - an answer to: "what are...

## The Binary Problem and The Continuous Problem in A/B testing

Posted by **Cameron Davidson-Pilon** at

Introduction I feel like there is a misconception in performing A/B tests. I've seen blogs, articles, etc. that show off the result of an A/B test, something like "converted X% better". But this is not what the A/B test was actually measuring: an A/B test is measuring "which group is better" (the binary problem), not "how much better" (the continuous problem). In practice, here's what happens: the tester waits until the A/B test is over (hence solving the binary problem),...