Probability & Expected Value

3.1.1 Every trade is a bet

Strip away the charts and the jargon and every trade is a bet: you put money at risk, several things can happen, and each has a probability. Quant work starts by writing the bet down precisely enough to compute with it.

Take a concrete trade. You risk a $10,000 stake on a breakout setup, and your journal (or a backtest) says it ends one of two ways: 40% of the time the breakout runs and you make $300; 60% of the time it fails and your stop costs you $150. The P&L of your next trade is unknown today, but you know which values it can take and how likely each one is. That is exactly what a random variable is: a number decided by chance, described by its distribution — the list of possible values and their probabilities. (Formally it is a function from outcomes, "the breakout runs", to numbers, $300: a function whose input nature chooses.)

Two numbers summarise a bet well enough to start. The expected value (EV) is the probability-weighted average outcome — what the bet pays per trade in the long run. The variance measures how far single outcomes stray from that average: it is the probability-weighted average of the squared distances from the EV. Its square root, the standard deviation, is back in dollars. In symbols, for a bet that pays $x_i$ with probability $p_i$:

For a continuous random variable with density $f$, the sums become integrals, $\E[X] = \int x\, f(x)\,\dd x$; nothing else changes. For the breakout trade, EV = 0.4 × $300 − 0.6 × $150 = $30 and the standard deviation is $220: the trade earns $30 on average, and a single trade typically lands about $220 away from that.

Now run a second setup through the same six steps: a mean-reversion fade that wins 70% of the time, making $100 when right and losing $250 when wrong. Most people prefer it at first sight — it wins almost twice as often. The arithmetic disagrees:

The last row is the habit this whole session builds: derive the answer, then simulate it and check that the two agree within four standard errors (the "±" is its standard error, §3.1.5). The lab does it like this:

def ev_and_var(outcomes: np.ndarray, probs: np.ndarray) -> tuple[float, float]:
    """Closed-form expected value and variance of a discrete bet."""
    ev = np.sum(probs * outcomes)
    var = np.sum(probs * (outcomes - ev) ** 2)
    return float(ev), float(var)

for name, (x, p) in BETS.items():
    ev, var = ev_and_var(x, p)
    draws = rng_bets.choice(x, size=N_SIM, p=p)          # simulate the same bet a million times
    sim_ev, sim_var = draws.mean(), draws.var()
    se_ev = np.sqrt(var / N_SIM)
    se_var = np.sqrt((central_moment4(x, p) - var ** 2) / N_SIM)
    assert abs(sim_ev - ev) < 4 * se_ev, f"bet {name}: simulated EV off by more than 4 SE"
    assert abs(sim_var - var) < 4 * se_var, f"bet {name}: simulated variance off by more than 4 SE"

3.1.2 Adding bets: linear EV, non-linear risk

A strategy is a pile of bets, and a portfolio is a pile of strategies. Expected values add up no matter how the bets are related; risks add up in a way that depends entirely on how they are related.

Linearity of expectation is the most useful fact in this session. For any random variables and constants,

— whether or not $X$ and $Y$ are related. It holds because an expectation is a weighted sum, and sums can be rearranged. So a year of bet A, one trade per trading day, has an EV of 252 × $30 = $7,560, with no assumption at all about whether one trade affects the next. Doubling the stake doubles the EV. Interviewers love linearity because it solves problems whose distributions are hopeless to write down (§3.1.8).

Variance is not linear. Scaling a bet by $a$ scales its variance by $a^2$, so doubling the stake quadruples the variance and doubles the standard deviation. Adding two bets brings in a cross term, the covariance, which is positive when the two tend to land above their means together:

Divide the covariance by both standard deviations and you get the correlation $\rho$, between −1 and 1. For $n$ bets that each have standard deviation $\sigma$ and pairwise correlation $\rho$:

With independent bets ($\rho = 0$) the variance is $n\sigma^2$ and the standard deviation grows only like $\sqrt n$, while the EV grows like $n$. That mismatch is the engine of §3.1.5. For the year of bet A it gives a standard deviation of $3,500 around the $7,560 EV. The chance of a losing year is 1.5% by the normal approximation of §3.1.6, and 1.24% exactly (the year loses money if 83 or fewer of the 252 trades win, a binomial probability) — close, but a first taste of how approximations drift in the tails.

Correlation changes the risk but not the EV. The lab holds 10 copies of bet A at once — ten stocks with the same breakout setup on the same day — and simulates 200,000 days, first with independent positions and then with a pairwise correlation of 0.3 (on a random 30% of days all ten share one coin flip, which gives exactly that correlation):

A modest correlation makes the same ten bets 1.92 times as risky, for exactly the same EV. At that correlation the ten positions carry the variance of about 37 independent ones — not ten.

3.1.3 Conditioning & independence

New information changes probabilities by shrinking the world: you throw away the outcomes that can no longer be true and re-weigh the rest. That one move, conditioning, sits behind every signal, every risk model and Bayes' rule.

The conditional probability of A given B is the share of B's world in which A also happens:

Read it as a two-step recipe: keep only the outcomes where B happened, then rescale their probabilities so they sum to 1 again. Fig 3.1.2 draws it for a trading question you will meet properly in §3.1.4: once you know the model said "long", the "short" days are gone, and the probability of an up-day is the up-share of what is left.

The same move works for averages. The conditional expectation $\E[X \mid B]$ is the average of $X$ over the shrunken world, and averaging the conditional averages back gives the law of total expectation: for events $B_1, \dots, B_k$ that split the world into non-overlapping pieces,

In trading terms: a strategy's average P&L is the frequency-weighted average of its P&L in each state — signal long or short, calm market or stormy. You will use this in §3.1.4 to price a signal.

Independence, and why zero correlation is not enough

Two events are independent when $\Prob(A \cap B) = \Prob(A)\,\Prob(B)$ — equivalently $\Prob(A \mid B) = \Prob(A)$: learning B does not change your odds on A. Two random variables are independent when every event about one is independent of every event about the other. Independence implies $\E[XY] = \E[X]\,\E[Y]$, hence zero covariance. The converse fails. Let $X$ be symmetric around zero, say standard normal, and $Y = X^2$. Then $\Cov(X, Y) = \E[X^3] - \E[X]\,\E[X^2] = 0$: the two are uncorrelated, yet $Y$ is a function of $X$ — tell me $X$ and I will tell you $Y$ exactly. Correlation only measures linear association.

Markets serve up this counterexample every day. The lab builds a toy market with two volatility regimes: calm days with a daily standard deviation of 0.8% and stormy days with 2.4%. Regimes persist — a calm day turns stormy with probability 2%, a stormy day calms with probability 8% — so 20% of days are stormy in the long run. The sign of each return is a fair coin. Over 1,000 simulated 10-year histories:

• The correlation between today's and tomorrow's return is +0.0005 ± 0.0010 — zero, as theory says. Today's direction tells you nothing about tomorrow's.
• But a "big" move, beyond ±3%, happens on 4.2% of days overall (simulated 4.2%) and on 19.4% of days that follow a big move (simulated 19.5%) — 4.6 times as likely. The correlation between consecutive squared returns is 0.21.

Real returns behave the same way: daily returns show almost no linear autocorrelation, while the autocorrelation of their absolute values is positive and decays only slowly with the lag — volatility clustering, one of the stylized facts in Cont's (2001) survey, which Session 5.1 measures and Session 5.5 forecasts.

3.1.4 Bayes' rule: the model said long

Whoever builds a signal measures how often it fires when the market goes up. Whoever trades it needs the reverse: how often the market goes up when it fires. Bayes' rule converts one into the other.

Here is a deliberately strong toy classifier. Suppose 52% of days are up days. On up days the model says "long" 70% of the time; on down days it still says "long" 55% of the time. Its builder reports, truthfully, "the model catches 70% of up days". This morning it says long. What is the probability that today is an up day?

The tempting answer is 70%. It is wrong, because that number is $\Prob(\text{long} \mid \text{up})$, the probability of the signal given the direction, and you want $\Prob(\text{up} \mid \text{long})$. Of all the days on which the model says long, some are up days (the green leaf of fig 3.1.3) and some are down days (the coral leaf); the answer is the green share. Writing each leaf as a product of its branches gives Bayes' rule:

In general, for a hypothesis $H$ and data $D$, $\Prob(H \mid D) = \Prob(D \mid H)\,\Prob(H) / \Prob(D)$, where the denominator is the law of total probability — the same "split the world and add up" move as the law of total expectation. The three pieces have names you will meet again: the prior $\Prob(H)$, the likelihood $\Prob(D \mid H)$ and the posterior $\Prob(H \mid D)$. The rule is named after Thomas Bayes, whose posthumous essay solved a special case of it (Bayes 1763).

People reason about this far more accurately in counts than in probabilities (Gigerenzer & Hoffrage 1995), so count. Out of 1,000 days, 520 are up and the model says long on 364 of them; 480 are down and it says long on 264 of them. It says long on 628 days, and 364 of those are up: 58.0%, not 70%. The lab checks every number against 1,000,000 simulated days, with the size of each day's move drawn at random (average 1%) and independent of the signal once the direction is known:

Two lessons hide in the table. The short calls are as reliable as the long ones although the model catches only 45% of down days: poor recall on a class does not mean poor precision. And the last three rows are the law of total expectation: long days are 62.8% of all days, and weighting the two conditional averages by those frequencies recovers the unconditional +0.04%. A signal is worth what it earns when it speaks — here about 0.16% a day on each side, before costs.

3.1.5 The law of large numbers

An edge is a statement about averages, and an average needs many observations before it means anything. The law of large numbers promises that the average finds the edge; the √n rule tells you how long that takes.

Let $X_1, X_2, \dots$ be independent repetitions of the same bet, with average $\bar X_n = \tfrac{1}{n}\sum_{i=1}^{n} X_i$. The law of large numbers says that $\bar X_n$ converges to $\E[X]$: for any tolerance $\varepsilon \gt 0$, the probability that $|\bar X_n - \E[X]| \gt \varepsilon$ goes to zero as $n$ grows. Jacob Bernoulli proved the first version, for repeated trials with two outcomes, in his Ars Conjectandi, published in 1713 after his death (Bernoulli 1713/2006). It is the reason a casino sleeps well.

The law says nothing about speed; §3.1.2 does. The variance of an average of $n$ independent bets is $\sigma^2/n$, so its standard deviation — the standard error of the average — is $\sigma/\sqrt n$. That answers two different questions. First, when can luck no longer hide the edge? The band of ±1.96 standard errors around the true edge stops covering break-even once $\E[X] \gt 1.96\,\sigma/\sqrt n$, that is, after

Second, when will your own test confirm it? Later — because your average is itself one draw from that band.

Put numbers in. A bet that wins $100 with probability 51% and loses $100 otherwise has an EV of $2.00 and a standard deviation of $99.98: the edge is 2% of one bet's noise. The formula gives $n^\star$ = 9,600 bets — about 38 years at one bet per trading day. Fig 3.1.5 follows three simulated players for 20,000 bets each. For the first ~9,600 bets the band still straddles break-even, so a player who quit early could have "proved" either that the bet is a winner or that it is a loser. After 20,000 bets they sit at $2.98, $0.83 and $2.38 against a true $2 — closer, but still visibly scattered.

After $n^\star$ bets the band of luck around the edge no longer covers break-even, so about 97.4% of players are ahead. A player who earned exactly the edge would just pass a 5% significance test, but only half of players earn that much or more: at $n^\star$ the chance that a player's own test confirms the edge is 50%. For an 80% chance you need about $n_{80} = \big((1.96 + 0.84)\,\sigma/\E[X]\big)^2$ = 19,614 bets — 78 years of daily bets. The reverse also happens: lucky players can pass long before $n^\star$. The last column of the table counts both effects.

Expected profit grows like $n$, luck like $\sqrt n$, so the ratio grows like $\sqrt n$ — the whole law in one line. This is the arithmetic behind "how many years before a Sharpe ratio means anything" in Session 1.4: a Sharpe ratio is an EV divided by a standard deviation. It is also why quants prize breadth (Session 8.7): 9,600 independent bets take 38 years on one market, or about a year spread across 38 unrelated ones — if they really are unrelated, which §3.1.2 says to check. And it is why simulation works: an average of $n$ simulated draws sits within a few $\sigma/\sqrt n$ of the truth, the Monte Carlo standard error of Session 3.5.

3.1.6 The central limit theorem, and when it is slow

The law of large numbers says where the average ends up. The central limit theorem says what its errors look like on the way — a bell curve, eventually. "Eventually" can take a very long time when the tails are fat.

The central limit theorem (CLT): if the $X_i$ are independent draws from any distribution with finite variance $\sigma^2$, the standardised average

has a distribution that approaches the standard normal distribution as $n$ grows, whatever the shape of the original bet. That is why the normal is everywhere: every "±1.96 standard errors" interval, every t-statistic and the 1.5% chance of a losing year in §3.1.2 lean on it. A standard normal lands within ±1 with probability 68.3% and beyond ±3 with probability 0.27%; those are the two yardsticks below.

The theorem is a limit, and limits hide their speed. The lab averages $n$ draws 100,000 times for two distributions. One is uniform — thin-tailed, bounded, as tame as it gets. The other is a Student-t with 2.5 degrees of freedom, a stand-in for fat-tailed returns: its variance is finite (5), so the CLT applies, but rare huge draws carry much of that variance.

The uniform is close to normal by $n$ = 10 and indistinguishable from it, at this sample size, by $n$ = 30. The Student-t is not normal at $n$ = 1,000: its averages still land beyond ±3 standard errors 2.0 times as often as the normal predicts, and pile up too tightly in the middle, because a few enormous draws account for much of the variance. There is a theorem behind the contrast. When the third absolute moment $\E|X - \E[X]|^3$ is finite, the Berry–Esseen theorem (Berry 1941; Carl-Gustav Esseen proved it independently) bounds the largest gap between the distribution of $Z_n$ and the normal by a constant times $\E|X - \E[X]|^3 / (\sigma^3 \sqrt n)$. A Student-t with 2.5 degrees of freedom has no finite third moment, so that $1/\sqrt n$ guarantee does not apply, and convergence is visibly slower.

3.1.7 Expected value is not what you experience

Expected value averages across parallel worlds. You live in one world, one bet after another — and when each bet stakes what the last one left you, the two averages part ways.

Peters & Gell-Mann (2016) made the point with a coin. Heads, your wealth grows by 50%; tails, it shrinks by 40%. Every round you stake everything you have. The EV of one round is +5%, and because the rounds are independent, expected wealth grows by that factor every round: after 20 rounds it is $2.65 per dollar. The lab gives the coin to 100,000 players for 20 rounds:

The EV is not wrong; it is answering a different question. Wealth multiplies: $W_t = W_0 \prod_{i=1}^{t}(1 + R_i)$. Taking logs turns the product into a sum — the decibel view of log returns from Session 2.3 — and the law of large numbers applies to the sum:

For this coin the expected log growth per round is $\E[\ln(1+R)] = \tfrac12\ln 1.5 + \tfrac12\ln 0.6 = \tfrac12\ln 0.9$, which is −0.0527: over many rounds, almost every player's wealth compounds at about −5.1% a round. The mean across players is carried by a handful who flip many more heads than tails (the order does not matter, only the count). Session 2.3 gave you a shortcut for the same gap, the volatility drag: the round's return has mean 5% and standard deviation 45%, so $\mu - \sigma^2/2$ = −0.0512, close to the exact log growth of −0.0527.

3.1.8 Interview classics, solved twice

Quant interviews open with dice, coins and cards because they test exactly this session's tools under time pressure. Solve each one exactly, then check it with a simulation: the habit is worth more than any single puzzle.

1. A die with one re-roll

   You are paid the face value of a fair die, and after seeing the first roll you may re-roll once. What is the game worth? A re-roll is worth 3.5, so keep a first roll of 4, 5 or 6 (average 5) and re-roll 1 to 3: ½ × 5 + ½ × 3.5 = 4.25. The tool is step 6 of fig 3.1.1: compare each outcome with the EV of the alternative — the same logic as deciding whether to hold a position or exit.

2. HH versus HT

   Flip a fair coin until HH appears; how many flips on average? And for HT? Condition on what you have just seen. For HT, wait for the first H (2 flips on average), after which the next T finishes: 2 more, so 4. For HH, a T after an H sends you back to the start. Let $a$ be the expected flips from scratch and $b$ from "just saw H": $a = 1 + \tfrac12 b + \tfrac12 a$ and $b = 1 + \tfrac12 a$, so $a = 6$. Same coin, same length, different answers: a failed HT attempt (another H) still leaves you holding an H, while a failed HH attempt (a T) throws away your progress.

3. The first ace

   Shuffle a deck. On average, at which position is the first ace? Label the 48 other cards. Each one is equally likely to fall anywhere among itself and the four aces, so it comes before all four with probability 1/5. By linearity, the expected number of cards before the first ace is 48/5 = 9.6, so the first ace sits at 10.6 on average — without ever writing down a distribution.

4. A coin from a bag

   One of ten coins has two heads. You pick one at random, flip it three times and see three heads. How likely is it the two-headed coin? Bayes: prior 1/10, likelihoods 1 and 1/8, posterior (1/10) ÷ (1/10 + 9/10 × 1/8) = 8/17.

Each simulation runs 400,000 trials and asserts agreement within four standard errors. The largest gap among the five is 2.8 standard errors — normal for honest simulation: across thirty-odd checks a gap near 3 turns up somewhere, which is why the asserts allow four (and why you never re-run with new seeds until a check passes). The HH and HT counts come from one small, vectorised function:

def flips_until(pattern: str, n: int, rng: np.random.Generator) -> np.ndarray:
    """Number of fair-coin flips until `pattern` (two letters of H/T) first appears, for n players."""
    want_first, want_second = pattern[0] == "H", pattern[1] == "H"
    count = np.zeros(n, dtype=int)
    prev = np.zeros(n, dtype=bool)          # was the previous flip heads?
    done = np.zeros(n, dtype=bool)
    while not done.all():
        flip = rng.random(n) < 0.5          # True = heads
        live = ~done
        count[live] += 1
        done |= live & (count >= 2) & (prev == want_first) & (flip == want_second)
        prev = np.where(live, flip, prev)
    return count

Session 16.2 turns them into timed drills with the market-making games that follow, and Mosteller's (1965) short book of classic puzzles, each with a worked solution, is the best single collection in the same spirit.

3.1.9 Try it

Predict first, then run. Each experiment in the lab has its own random stream, so changing one leaves the others untouched.

3.1.10 Glossary · Check yourself · Go deeper

Check yourself

📚 Go deeper

1. Blitzstein, J. K. & Hwang, J. (2019). Introduction to Probability, 2nd ed. Chapman & Hall/CRC. Chapters 2 (conditional probability), 4 (expectation), 9 (conditional expectation) and 10 (inequalities and limit theorems). ISBN 978-1-138-36991-7
2. Mosteller, F. (1965). Fifty Challenging Problems in Probability with Solutions. Addison-Wesley; Dover reprint 1987. ISBN 978-0-486-65355-6
3. Peters, O. (2019). "The ergodicity problem in economics." Nature Physics 15(12), 1216–1221. doi:10.1038/s41567-019-0732-0
4. Gigerenzer, G. & Hoffrage, U. (1995). "How to Improve Bayesian Reasoning Without Instruction: Frequency Formats." Psychological Review 102(4), 684–704. doi:10.1037/0033-295X.102.4.684

Sources for the numbers

1. Peters, O. & Gell-Mann, M. (2016). "Evaluating gambles using dynamics." Chaos 26(2), 023103. doi:10.1063/1.4940236
2. Bayes, T. (1763). "An Essay towards Solving a Problem in the Doctrine of Chances." Philosophical Transactions of the Royal Society of London 53, 370–418. doi:10.1098/rstl.1763.0053
3. Bernoulli, J. (1713/2006). The Art of Conjecturing, together with Letter to a Friend on Sets in Court Tennis, translated by E. D. Sylla. Johns Hopkins University Press. ISBN 978-0-8018-8235-7
4. Berry, A. C. (1941). "The Accuracy of the Gaussian Approximation to the Sum of Independent Variates." Transactions of the American Mathematical Society 49(1), 122–136. doi:10.1090/S0002-9947-1941-0003498-3
5. Cont, R. (2001). "Empirical properties of asset returns: stylized facts and statistical issues." Quantitative Finance 1(2), 223–236. doi:10.1080/713665670