Before changing it, put it to the test.
From hypothesis to decision. A training case with A/B tests.
Python · Statistics · A/B Testing
What changes.
Objective
Validate whether a new checkout flow improves conversion compared to control, maintaining statistical rigor.
Results
Significant difference favoring group B (p<0.05).
How it was built.Decisions, evidence and details
Objective
Validate whether a new checkout flow improves conversion compared to control, maintaining statistical rigor.
Methodology
- Preparation: clean visitor and order data, verify the A/B balance.
- Hypotheses: H0 assumes no difference; H1 expects conversion B > A.
- Testing: conversion rate, Welch t-test (p<0.05), and assumption checks (Shapiro, Levene).
- Visualization: histograms and boxplots of conversion by group.
Key visualizations
- Conversion distribution by group A vs. B.
- Daily trend of the conversion rate throughout the experiment.
- Summary of p-values and confidence intervals.
Results
- Significant difference favoring group B (p<0.05).
- Recommendation to deploy variant B and monitor for two more weeks.
- Reusable template for future A/B experiments.