Starting with the distribution
The habit that prevents most errors in this industry is looking at the distribution before quoting any summary of it.
For any measure, plot it. Revenue per customer, session length, deposit size, days between sessions. In almost every case the result will be heavily skewed: a large mass of small values and a long tail of large ones.
Once that is visible, several things follow automatically.
The mean sits well above the bulk of the observations and describes a customer who does not exist.
The median describes the typical customer and understates the total.
The deciles show where the value actually sits, which is usually startling the first time it is examined.
The tail contains a small number of customers who may individually matter more than large segments.
An analyst who has looked at the distribution will not quote an average revenue per user without context. One who has not will quote it, and the figure will be used as though it described someone.
Summary statistics for skewed data
Given the shape, the appropriate summaries.
Median for the typical value.
Mean for the total divided by the count, which is what it is, and which is useful for aggregate planning and useless for describing individuals.
Deciles or percentiles for the shape, and particularly the top few percentiles, which is where the concentration lives.
Trimmed mean, excluding a defined proportion at each extreme, which gives a central tendency less influenced by outliers.
Proportion of total from proportion of population, which is the concentration measure and among the most informative single figures available. What share of revenue comes from the top 1%, 5% and 10% of customers, and how has it moved.
Range and interquartile range rather than standard deviation, since the latter assumes a shape this data does not have.
The presentational discipline is that a mean should rarely appear alone. Paired with a median it becomes informative, since the gap between them is itself a description of the skew.
Cohort analysis
The most useful technique available, and the one that answers the questions aggregate reporting cannot.
A cohort is a group defined by a shared starting point, usually the period in which they were acquired. Cohort analysis tracks each group separately over time and compares groups at equivalent ages.
The value is that it separates two effects that aggregate reporting conflates: the effect of a customer's age in the relationship, and the effect of when they were acquired.
The classic finding this enables was described in the iGaming Basics course. Aggregate revenue rises steadily while each successive cohort is worth less than its predecessor, because enough new customers arrive to mask the decline. In aggregate this looks like growth. In cohort view it is deterioration with a lag.
Constructing a cohort analysis. Define the cohort by acquisition period, usually month. Define the measure, usually contribution or retention. Define the age intervals, usually months since acquisition. Then produce a matrix with cohorts as rows and ages as columns.
Reading it. Look down a column to compare cohorts at the same age, which reveals whether acquisition quality is changing. Look across a row to see a single cohort's trajectory. Look diagonally to see what happened in a specific calendar period across all cohorts, which reveals external effects.
Retention curves plot the proportion of each cohort still active at each age. In this industry the curve is steep initially and flattens, which is why early tenure matters disproportionately.
Contribution curves plot cumulative contribution per customer, which combined with acquisition cost gives payback period.
The traps. Cohorts must be large enough to be stable, which given the skew means larger than intuition suggests. The definition of active must be held constant. And the comparison must be at equivalent ages, since a recent cohort has not had time to accumulate what an older one has.
Segmentation with skew
The considerations specific to this data, extending the CRM material.
Value segmentation must use deciles or similar rather than fixed thresholds, since the distribution moves and fixed bands become unbalanced.
The top segment may be tiny and carry most of the value, which means a segment containing a fraction of a percent of customers may warrant its own analysis rather than being folded into a top decile.
Segment sizes should be reported alongside segment metrics, since a striking figure from a segment of forty customers is not a finding.
Mix shifts drive aggregate changes. A change in an aggregate figure may reflect a change in the composition of the population rather than a change in behaviour within any segment, which is the paradox described below.
Segments should be defined on stable characteristics where possible, since segmenting on a measure that moves means customers migrate between segments and the segments' composition changes underneath the analysis.
Simpson's paradox
The specific trap worth naming, because it appears regularly in this data and produces confident wrong conclusions.
An aggregate trend can be the opposite of the trend within every segment, when the mix between segments changes.
A worked example. An operator's average revenue per customer falls. Examined by segment, average revenue per customer has risen in every value tier. Both are true, because the operator acquired a large number of low-value customers, which increased the proportion of the base in the lowest tier and pulled the aggregate down while every tier improved.
The aggregate says performance deteriorated. The segments say it improved. Which conclusion is right depends on the question: revenue per customer genuinely fell, and per-customer performance within each group genuinely rose, and an operator that reads only one has an incomplete picture.
The practical rule is that any aggregate finding is provisional until the segments have been checked, and that a change in an aggregate should prompt the question of whether the population's composition changed.
Comparison and its pitfalls
Given skew, comparing two groups requires care.
Check the composition before comparing. Two groups with different value distributions are not comparable on averages regardless of how they were selected.
Compare like ages in cohort work, since a cohort acquired last month has not had time to do what one acquired a year ago has done.
Compare within segments as well as in aggregate, given the paradox above.
Check whether the difference survives outlier removal, which is the most useful single check available.
Use the median as well as the mean, since a difference present in one and absent in the other is informative about where it came from.
Be cautious about small groups, since with a skewed distribution a small group's mean is dominated by whether it happened to contain a large customer.
Watch for selection. Groups formed by behaviour are not comparable to groups that did not exhibit that behaviour, which is the causal problem the next lesson develops.
Time series considerations
A few characteristics of this data worth knowing.
Strong weekly seasonality in most verticals, with weekends and evenings dominant, which means any comparison must align on day of week.
Sporting calendar effects in sportsbook that dominate everything else, with major tournaments producing multiples of baseline activity.
Payday effects visible in deposit patterns in most markets.
Holiday effects that differ by market and can be substantial.
Promotional periods that create artificial peaks and subsequent troughs, meaning a comparison spanning a campaign measures the campaign.
Regulatory change points, after which comparison to prior periods may be invalid because the product changed.
The practical guidance is to compare like periods, to be explicit about what a period contained, and to be sceptical of any short-period comparison in a business with this much systematic variation.
Practical checks
To close, a routine to apply to any analysis before presenting it.
Have I looked at the distribution, or only at a summary of it?
Does the effect survive removing the largest few observations?
Do the median and the mean agree on the direction of the finding?
Are the groups comparable in composition, not only in size?
Does the aggregate finding hold within segments?
Are the periods comparable, on day of week, seasonality and what they contained?
Is the sample large enough given the variance in this data, which is higher than most consumer data?
Could this be a mix shift rather than a behaviour change?
Would I expect this result? And if it is surprising in a convenient direction, what would I check before believing it?
Nine checks, each taking minutes, and between them they catch the great majority of the errors that skew produces in this industry.
A worked cohort analysis
To make the technique concrete, an operator examines whether its acquisition is improving.
Aggregate revenue has grown 18% year on year. Active players have grown 24%. The commercial team reports a successful year.
The cohort view is constructed: customers grouped by acquisition month, contribution tracked by months since acquisition.
Reading down the third-month column across cohorts shows contribution per customer at month three falling steadily. The cohort acquired eighteen months ago generated a given figure by month three. The cohort acquired three months ago generated substantially less.
Reading across a row shows each cohort's trajectory is similar in shape, so the decline is not about how customers behave over time.
Reading the diagonal shows no calendar-period effect explaining it, so it is not an external event.
The finding is that acquisition quality has deteriorated steadily while volume increased enough to mask it in aggregate. Revenue grew because more customers were acquired, each worth less than their predecessors.
Extending the analysis by acquisition source locates it: the deterioration concentrates in a channel that was scaled up during the year, and the cohorts from other sources are stable.
Adding acquisition cost shows the payback period on that channel lengthening across the year.
The conclusion changes the decision entirely. Aggregate reporting supported increasing spend on a channel delivering growth. The cohort view shows that channel delivering progressively worse customers at progressively worse payback, with the aggregate improvement funded by volume that will stop working.
That analysis takes an afternoon given the data, is unavailable in aggregate reporting, and is the single most valuable routine analysis in this industry.
Working with small numbers
A practical problem that arises constantly given skew.
Segments of interest are frequently small. The top percentile of customers. Players of a specific game. Customers in a market at launch. A cohort in an early month.
With small groups and a skewed distribution, almost nothing is stable. A group of two hundred customers has a mean dominated by whether it contains a very large one, and month-to-month movement will be substantial and meaningless.
The disciplines that help.
Report the group size alongside any figure from it, always.
Prefer counts and proportions to averages in small groups, since these are more stable.
Aggregate periods to build sample, accepting reduced timeliness.
Use medians, which are considerably more stable than means in small skewed samples.
Show the individual observations where the group is small enough, since a scatter of forty points conveys more than a summary of them.
State the uncertainty explicitly, since a reader given a figure from a group of forty will treat it with the same confidence as one from a group of forty thousand unless told otherwise.
Resist the temptation to segment further, since each additional dimension divides the sample and at some point produces cells too small to mean anything.
Building the routine analyses
To close, the analyses that should exist as standing capability rather than being constructed each time.
Cohort retention curves by acquisition month, updated monthly, with the ability to split by source and market.
Cohort contribution curves, the same, against acquisition cost, giving payback.
Value distribution by decile and by top percentiles, with the trend.
Concentration measures, meaning the proportion of revenue from the top segments, which serves both commercial and protective purposes as the metrics lesson described.
Funnel completion by stage, market and device.
Session and spend distributions, including the tails.
Churn by tenure, showing where in a customer's life the risk concentrates.
Product mix by segment, showing what different customers actually do.
Each of these answers a recurring question. Building them once as maintained assets, rather than reconstructing them for each request, is what allows an analytics function to spend its time on the questions that are not recurring.
The related discipline is that these should be actively reviewed rather than merely available. A cohort curve that nobody has looked at for six months is not providing information; it is providing the possibility of information, which is not the same thing.
Why the skew exists
A closing point of understanding, since knowing why the distribution has this shape helps in reasoning about it.
Several mechanisms compound.
Variation in interest. Some people enjoy gambling considerably more than others and engage accordingly, which produces variation before any other factor applies.
Variation in means. Disposable income varies enormously across a customer base, and stake sizes follow.
Variation in tenure. A customer of three years has had far more opportunity to accumulate spend than one of three weeks, and any cross-sectional view mixes them.
Session frequency compounding. Small differences in how often someone plays produce large differences in totals over time.
Product mix. Higher-stake products attract a subset of customers, and their contribution scales differently.
Retention selection. Customers who remain are disproportionately those who enjoy it most, which means the surviving population is more concentrated than the acquired one.
The mechanisms are ordinary and the resulting distribution is not an anomaly to be corrected. It is the shape of the business, and analysis should be built for it rather than applying methods that assume something else.
The uncomfortable corollary, made in the iGaming Basics course and worth restating in an analytical context, is that the concentration which makes averages misleading is the same concentration that makes the protective questions pressing. An analyst producing a decile breakdown for commercial reasons has produced a figure that also indicates how dependent the operator is on a small number of very heavy spenders, and the second reading is available at no additional cost.
Checking your own work
A short habit worth building, applied before any finding leaves the analyst.
Re-derive the headline figure by a different route. If two methods agree the number is probably right; if they disagree the reason is worth knowing before anyone else asks.
Sanity-check magnitudes against something known. A revenue figure should be consistent with what finance reports; a customer count with what the platform holds.
Look for round numbers and repeated values, which frequently indicate a data problem rather than a real pattern.
Check the boundaries, meaning the first and last periods, which are where truncation and incomplete data appear.
Ask what would make this wrong and check that specific thing.
The general principle is that an analyst is the last person who will scrutinise the work with full knowledge of how it was constructed, and that a check taking ten minutes is considerably cheaper than a correction after the finding has been acted on.