1.4 · The Engineer's Edge, Blind Spots & Honest Odds

1.4.1 Two worlds

Everything you know about building software was learned in a world that is deterministic, cooperative and quick to tell you when you are wrong. Markets are none of those three, and each difference breaks a habit.

fig 1.4.1 — the world you trained in and the world you are entering, row by row

Read the diagram row by row. A failing unit test fails every run; a genuinely profitable trading rule still loses money in about 21% of years (§1.4.4). A model that explains most of its target is ordinary in ML; in markets, explaining a quarter of one percent of tomorrow's return is a real edge (§1.4.3). Your production system does not change because you shipped a fix; the market does, because the traders on the other side of your edge lose when you win, and they adapt. And you cannot generate more history: twenty years of daily prices is 5,040 observations from a process that will never repeat.

The whole session in one sentence: software is an engineering problem with a spec; trading is a statistical problem with an opponent. Your engineering skills still matter enormously. They are just not sufficient, and a few of them point the wrong way.

📌 Key idea — your tests can pass by luck

Session 1.3 — The Quant Research Lifecycle compared a backtest to a unit test that can pass by luck. This session measures how much luck. In markets, a green test is evidence, not proof, and how much evidence depends on the noise, the time and the number of tries.

🧭 Margin note — the analogies we will keep using

A backtest is a unit test that can pass by luck. Look-ahead bias is train/test leakage. A holdout set is a one-time password. Non-stationarity is an upstream API that changes its behaviour without a changelog. And the market is a counterparty who is paid to break your code.

1.4.2 Your edge, and your blind spots

Not everything breaks. Much of what you do every day is exactly what quant teams need, while a few habits feel right and quietly wreck research.

fig 1.4.2 — the engineer's edge and blind-spot matrix: the dangerous quadrant is the one that feels like competence

The top-left quadrant is real. Many ways a backtest lies are plain bugs: a date misaligned by a day, a price you could not have traded, returns summed where they should compound (Session 2.3 lists six such bugs). An engineer who writes tests catches them. The bottom-left is edge you may not notice: if you have cut a cloud bill, you already think the right way about transaction costs; blameless post-mortems are a research log; canary releases are paper trading followed by small live size. The bottom-right is the subject of most of this session: habits that mark competence in your old world and quietly destroy research in the new one.

What you bring	Where it pays in quant work	Where the course uses it
Testing, CI, code review	Catching look-ahead, misaligned dates and return bugs before they become "alpha"	Session 2.3, Session 6.4
Data pipelines	Point-in-time, survivorship-free research data	Unit 4
Reproducibility: git, lockfiles, seeds	Research logs and pre-registrations someone can audit	Session 1.3, Session 1.5, Session 6.8
ML tooling	Gradient boosting and feature importance, once validation respects time	Session 6.6, Unit 12
Cost discipline	Trading costs and capacity, the silent killers of small edges	Session 6.5

💡 Looking ahead

Which quadrant matters most depends on your role (Session 1.1 — What Quants Actually Do). A quant developer lives mostly in the top-left. A researcher or independent trader has to clear the bottom-right first, because a blind spot there does not crash; it produces a strategy that looks excellent and loses money.

1.4.3 Signal-to-noise: why 0.25% can be an edge

The biggest culture shock for ML engineers: a forecast that explains a quarter of one percent of tomorrow's return can be worth a career, and one that claims several percent is almost certainly fooling you.

The signal-to-noise ratio of a prediction problem is how much of the outcome's variation the inputs can explain compared with how much they cannot. For images it is high; for daily returns it is tiny, because prices already reflect what most people know. Quants measure a forecast by its information coefficient (IC), the correlation between forecast and outcome; with one predictor, the share of variance explained is $R^2 = \mathrm{IC}^2$ .

fig 1.4.3 — what "signal" looks like in returns, drawn to scale: honest bars are a pixel or two wide; a wide one fitted in-sample should worry you

The arithmetic of a small edge

The lab builds a signal known before each day starts and a daily excess return (risk-free rate 0) that is a little signal and mostly noise, with correlation exactly $\rho =$ 0.05. It holds $s_t$ units of the asset on day $t$ : long when the signal is positive, short when negative, bigger when stronger. With $\sigma$ the daily volatility:

R_t = \sigma\big(\rho\, s_t + \sqrt{1-\rho^2}\;\varepsilon_t\big), \qquad s_t,\ \varepsilon_t \sim N(0, 1), \qquad R^2 = \rho^2

The daily profit $s_t R_t$ has mean $\rho\sigma$ and standard deviation $\sigma\sqrt{1+\rho^2}$ , which gives the annualised Sharpe ratio and the chance of calling the day's direction right:

\SR = \frac{\rho\sqrt{252}}{\sqrt{1+\rho^2}} \approx \rho\sqrt{252}, \qquad \Prob(\text{right direction}) = \tfrac12 + \tfrac{1}{\pi}\arcsin\rho

So the signal explains 0.25% of each day's return and calls its direction right 51.6% of the time, barely better than a coin. Yet traded every day it earns a Sharpe ratio of 0.79 before costs. The $\sqrt{252}$ is the whole trick: many small, independent bets add up. That is the fundamental law of active management, which the Picker exploited in Session 1.2 — Where Returns Come From. Robert Carver, a former portfolio manager at the systematic hedge fund AHL, puts it the practitioner's way: an $R^2$ of 0.01 on 5-day returns, which "sounds rubbish", is a Sharpe ratio of around 0.7 (Carver 2025).

# from the companion lab, section 1
def make_signal_and_returns(n_days, ic, rng, vol=DAILY_VOL):
    s = rng.standard_normal(n_days)                # known before the day starts
    noise = rng.standard_normal(n_days)
    R = vol * (ic * s + np.sqrt(1 - np.square(ic)) * noise)
    return s, R

s, R = make_signal_and_returns(DAYS * 20, IC_TRUE, rng_snr)
pnl = s * R                                        # hold s units of the asset each day
sharpe = annual_sharpe(pnl)                        # from quantnb.stats

Across 200 independent 20-year histories the average Sharpe ratio is 0.83 and the average hit rate 51.7%, matching theory within sampling error. The spread is sobering: twenty years of the same edge measured anywhere from 0.25 to 1.44.

Real research agrees on the scale. Gu, Kelly & Xiu (2020) applied modern machine learning to nearly 30,000 US stocks over 60 years. Their best model, a three-layer neural network, explained 0.40% of monthly single-stock returns out of sample; a long-short portfolio on a four-layer network's forecasts earned an out-of-sample Sharpe ratio of 1.35 before trading costs, with turnover of more than 100% a month.

The kitchen sink

Now the ML reflex: add features. The lab fits least squares on 5 years (1,260 days), scores it on the next 5, and repeats 200 times, with three menus: the real signal; the real signal plus 49 features of pure noise; and 50 noise features alone.

fig 1.4.4 — in-sample and out-of-sample R² for the real signal alone and with junk features added: the better fit to the past predicts the future worse than a forecast of zero

Model	R², 5 years fitted	R², 5 unseen years	Sharpe ratio, unseen years
1 real signal	0.34%	0.20%	0.76
Real signal + 49 junk	4.21%	−3.93%	0.17
50 junk features	3.96%	−4.25%	−0.04

This is overfitting: fitting noise in the sample as if it were signal. With $K$ useless features and $T$ observations, least squares buys an in-sample $R^2$ of about $K/T$ , here 4.0%, and pays about the same back out of sample. In 100% of repeats pure noise fitted the past better than the genuine signal, and adding the junk cut the real edge's out-of-sample Sharpe ratio from 0.76 to 0.17. Gu, Kelly & Xiu saw the same at scale: least squares on all 920 of their features scored an out-of-sample $R^2$ of −3.46%, worse than predicting zero.

📌 Key idea — small and real beats big and fake

Judge a return forecast by what it earns per unit of risk out of sample, not by how well it fits. In returns, a large in-sample fit is evidence of overfitting, not of skill.

1.4.4 Moving rules, live opponents, and a new loop

Three more differences break assumptions your ML training made for you, and together they force a different development loop.

The rules change while you play

Non-stationarity means the process generating the data changes over time. The formal idea, stationarity, belongs to Session 5.4 — Stationarity & ARMA Models; ML engineers know the symptom as concept drift. Markets move between regimes, and an edge can weaken, vanish or flip sign. The lab gives one signal an IC of 0.05 for 8 trading years (regime A, true Sharpe 0.79), then −0.03 for 4 (regime B, true Sharpe −0.48), as if others had crowded in and overshot.

fig 1.4.5 — the rolling one-year Sharpe ratio of one signal whose true Sharpe (grey) flips from 0.79 to −0.48 after year 8: the noise is as big as the break

In regime A the edge was real, yet the rolling Sharpe ratio fell to −0.7; in regime B it was negative, yet the rolling Sharpe rose to 1.5. Over thousands of simulated years the regime-B edge makes money in 32% of them. And the whole twelve years blend into one respectable number that hides the break: across 500 simulated histories the full-sample Sharpe ratio averages 0.38 (theory 0.37).

fig 1.4.6 — growth of $1 on the same history: $2.74 after year 8, $2.09 at the end — plain in hindsight, ambiguous while it happens

⚠️ Pitfall — "It lost money this year, so the edge is broken"

Or its twin, "it made money, so it's fine." A one-year Sharpe ratio has a standard error of about 1, bigger than almost any real edge. Decide in advance what evidence retires a strategy (a kill criterion, Session 1.3), check whether its economic story still holds, and use the monitoring tools of Session 9.9 — Performance, Attribution & When to Switch Off.

Your profit is someone's loss, and one history is all you get

Short-horizon trading is close to a zero-sum game before costs, so whoever pays for your edge has every reason to stop. Markets are an adversarial environment (Session 12.1) and a reflexive one: when enough people trade a pattern, their trading erases it, the alpha decay you met in Session 1.3.

And you cannot rerun 2008 with a different seed. Twenty years of daily data is 5,040 observations; a monthly strategy has 240. Israel, Kelly & Moskowitz (2020) name small data and a low signal-to-noise ratio among the reasons finance is harder for machine learning than the domains where it has excelled, and conclude that it must be used together with economic theory and human expertise.

So the loop must change

fig 1.4.7 — from build-test-ship to hypothesis, pre-register, explore, test once: iteration moves out of the test and into a logged, in-sample sandbox

The top row works because a unit test is a fixed, deterministic oracle. A backtest is not: every time you look at its result and change something, some of that history's noise leaks into your strategy, and enough loops reach green on any history, as the skill-less strategies of Session 1.3 showed. López de Prado (2018) lists "research through backtesting" among the ten reasons most machine-learning funds fail. The bottom row keeps the iteration but moves it in-sample, logs every trial so the winner can be deflated (Session 6.7), and runs the final test once against a criterion written down first (Session 1.3). A failed test kills the idea, because an out-of-sample set you have looked at is in-sample.

Habit from software or ML	Why it breaks in markets	Do this instead
Iterate until the test is green	The backtest becomes your training set; you ship the luckiest variant	Pre-register, log every trial, deflate the winner (Session 6.7, Session 6.8)
Random k-fold cross-validation	Returns and labels overlap in time, so shuffled folds leak the answer	Purged, embargoed, time-ordered validation (Session 6.6)
More features, bigger model	Fitted noise grows like $K/T$ (§1.4.3); complex products decay more live (§1.4.6)	Few features, each with an economic story (Session 1.3, Session 12.6)
Judge by accuracy or $R^2$	A real edge is right 51.6% of the time; a high in-sample $R^2$ is a red flag	Out-of-sample Sharpe ratio after costs (Session 8.7)
Treat costs as an ops detail	The lab's real edge breaks even at 4.5 bp per unit traded	Model costs from the first backtest (Session 6.5)

The cost row deserves its number. Holding $s_t$ units means trading $|s_t - s_{t-1}|$ units a day, 1.13 on average, against a gross edge of 5.1 bp a day. The Sharpe ratio of 0.80 before costs falls to 0.45 at 2 bp per unit traded and −0.09 at 5 bp: compute the break-even cost of every idea first.

⚠️ Pitfall — "Random k-fold cross-validation proves my model generalises"

Not on time series. If each label is a 20-day forward return, neighbouring labels share 19 days of price moves, so a shuffled fold hands the model near-copies of its test answers: leakage, and a broken IID assumption. Validate forward in time, remove overlapping samples from training (purging) and leave a gap after each test fold (an embargo), as Session 6.6 builds.

1.4.5 How long before you know?

The most sobering number in this session is how many years a track record needs before it means anything.

You met the key fact in Session 1.3: an estimated Sharpe ratio has a standard error of about $1/\sqrt{Y}$ after $Y$ years. For independent, normally distributed returns, Lo (2002) gives the general standard error on $Y$ yearly returns. Requiring the 95% interval to exclude zero (a Sharpe ratio at least 1.96 standard errors) gives the years needed:

\mathrm{SE}(\widehat{\SR}) \approx \sqrt{\frac{1 + \SR^2/2}{Y}}, \qquad Y \approx \Big(\frac{1.96}{\SR}\Big)^2\Big(1 + \frac{\SR^2}{2}\Big)

The engine is the $1/\SR^2$ : halve the Sharpe ratio and you need four times the data. The $\SR^2/2$ term is the price of also estimating the volatility; on daily returns it nearly vanishes, leaving $Y \approx (1.96/\SR)^2$ . Real returns have fat tails, skew and autocorrelation, which Session 6.7 adds back as the minimum track record length.

fig 1.4.8 — years of yearly returns before a Sharpe ratio is distinguishable from zero at 95% (Lo's formula): the realistic Sharpe ratios are the expensive ones. The formula is asymptotic, so the bars for high Sharpe ratios rest on only a few yearly observations

True Sharpe ratio	Years to 95%, yearly returns (Lo)	Years to 95%, daily returns
0.25	63.4	61.5
0.5	17.3	15.4
0.75	8.8	6.8
1.0	5.8	3.8
1.5	3.6	1.7
2.0	2.9	1.0

Now put realistic numbers in. Carver writes that Sharpe ratios consistently above 1.0 are rarely achieved, and suggests multiplying even a backtest fitted with out-of-sample methods by about three-quarters (Carver 2015). So you will live in the rows at 1.0 and below: roughly 4 to 15 years on daily data (6 to 17 on yearly returns) just to be 95% sure the Sharpe ratio is above zero. Even at those lengths, a strategy with exactly that Sharpe ratio passes only half the time (lab, section 4).

📌 Key idea — you will rarely be able to prove it

For the Sharpe ratios good strategies actually earn, a track record takes many years, often more than a decade, to separate skill from luck. Because the track record cannot prove a strategy in time, the process that produced it has to carry the weight: the method of Session 1.3.

💡 Looking ahead

Session 5.2 — Regression & Inference That Doesn't Lie derives this standard error, with corrections for autocorrelation; Session 16.4 — Running Your Own Capital uses the table to judge what your own track record can tell you.

1.4.6 The evidence: who actually makes money

Large samples, honestly measured. Read them as base rates for people in your position, not as verdicts on you.

Backtest-to-live decay is the drop in performance between a strategy's backtest and its live trading. It has three sources at once: overfitting, costs and frictions the simulation missed, and genuine erosion of the edge. Large samples find it everywhere, including among professionals:

Study	Sample	Finding	Read it as
Wiecki, Campbell, Lent & Stauth (2016)	888 Quantopian algorithms, 6+ months live	Backtest Sharpe explains under 2.5% of live Sharpe's variation; more backtesting, bigger gap	One backtest Sharpe is nearly uninformative
Suhonen, Lennkh & Perez (2017)	215 bank "alternative beta" strategies	Median Sharpe 73% lower live; the most complex decayed 30+ percentage points more than the simplest	Professionals overfit too, and complexity costs
Chague, De-Losso & Giovannetti (2020)	1,551 Brazilian futures day traders active 300+ days	97% lost money after fees; 1.1% earned more than the minimum wage	Day trading for a living is not a plan
Barber, Lee, Liu & Odean (2014)	Taiwan day traders, 1992–2006	Fewer than 1% predictably earn positive abnormal returns after fees	Skill exists; its base rate is tiny
Barber, Lee, Liu, Odean & Zhang (2020)	The same Taiwan data	97% likely to lose money in future day trading; most volume comes from past losers	Losing traders keep trading

A backtest's headline number predicts little, and less the harder its author searched; Harvey & Liu (2015) show that the right haircut is not a flat "halve it" but nonlinear, mild for very strong results and severe for marginal ones. And in the Brazilian data the share of day traders who made money fell the more days they traded: the pattern of a casino game, not a skill. Published anomalies decay too (Session 1.3).

Do not overgeneralise from day traders to everything systematic. Retail day trading is a short-horizon contest against professional market makers and high-frequency firms, negative-sum after costs. Systematic investing over months and years can also collect risk premia, paid by people who want to shed a risk (Session 1.2). That game has better base rates but is not easy: Suhonen's alternative-beta strategies are exactly such risk-premium products, and still lost most of their backtested Sharpe ratio. The evidence does not say "nobody makes money". It says the typical attempt fails, the typical backtest overstates, and what survives is modest.

⚠️ Pitfall — "Those studies are about amateurs. I'm an engineer."

The alternative-beta strategies came from global investment banks; the Brazilian traders kept at it for more than 300 days. The process that fooled them will fool you too unless you change the process, which is what the rest of this course is for.

1.4.7 Base rates: think like a forecaster

A good forecaster's first question is not "how good does this look?" but "how often does this kind of thing work?"

A base rate is how often an outcome happens in the reference class a case belongs to: the share of day traders who make a living, the share of backtests that survive live trading. Kahneman & Lovallo (1993) called starting from it the outside view, and argued that reasoning only from your own case (the inside view) produces over-optimistic forecasts. Tetlock & Gardner (2015) found that the best forecasters in a multi-year tournament start there and then adjust.

Apply it to your own research. Suppose a fraction $\pi$ of the ideas you test are real, each with a true Sharpe ratio of 0.5, and the rest have none. You test each once on 5 years of daily data and call it a discovery if its Sharpe clears 1.96 standard errors, about 0.88. A fake idea then passes 2.5% of the time, the test's size (statisticians write α, which this course reserves for alpha). A real one passes only 20% of the time, the test's power, because 5 years is far short of the 15 a Sharpe of 0.5 needs (§1.4.5). Bayes' rule gives the chance a passing idea is real, the argument Ioannidis (2005) used to explain why most published research findings are false:

\Prob(\text{real} \mid \text{pass}) = \frac{\pi \cdot \text{power}}{\pi \cdot \text{power} + (1 - \pi) \cdot \text{size}}

fig 1.4.9 — base rates as a tree: only two of the four boxes pass, and the formula is simply the green one's share of them

In natural frequencies: of 1,000 ideas at a 10% base rate, 100 are real and about 20 of them pass; 900 are not, and 22.5 of those pass by luck. Only 47% of the 42.5 passes are real: a coin flip. Simulating 200,000 ideas at each base rate confirms the formula:

Base rate: share of ideas that are real	P(real \| passed), Bayes	P(real \| passed), simulated
1%	7%	8%
5%	30%	30%
10%	47%	46%
25%	73%	73%
50%	89%	89%

That is the optimistic version, with one honest test per idea; trying many variants inflates the size (Session 1.3). The formula names your four levers:

Raise the base rate. Only test ideas that answer "who pays me, and why does it persist?" (Session 1.2, Session 1.3).
Raise the power. More independent bets, more assets, longer histories (breadth, Session 8.7). At a true Sharpe of 1.0 the same 5-year test has a power of 61%.
Shrink the size. Fewer, pre-registered tests, and results deflated for the trials you did run (Session 6.7).
Keep what you find. Model costs from day one (Session 6.5) and watch for decay (Session 9.9).

Your honest odds

So what are your odds? Name your reference class. Day trading for a living: the table in §1.4.6. A professional team, as a researcher or developer: you are paid to run this process with other people's data, capital and review, and a failed idea costs a lesson rather than your savings (Session 16.1 — Career Paths & the Quant Job Market). Your own systematic book: the odds depend almost entirely on process, with modest Sharpe ratios, honest validation, low costs, small size and years of patience (Session 16.4 — Running Your Own Capital). None is hopeless; all are harder than they look from the inside.

1.4.8 Try it

Predict first, then run. Each experiment has its own random stream, so changing one leaves the others untouched.

🔧 Try it (20 min, in the lab)

1. Before running anything, predict the Sharpe ratio and the break-even cost of a daily signal with an IC of 0.03, using $\SR \approx \rho\sqrt{252}$ and a break-even of $\rho\sigma$ divided by the daily turnover of 1.13. Write your numbers down.
2. Set IC_TRUE = 0.03 and rerun the lab. Compare your predictions with the 200-path average in section 1 and the break-even in section 3.
3. Restore it, set N_JUNK = 99 and rerun section 2. How much in-sample $R^2$ did the extra junk buy, and what happened out of sample?
4. In section 5, set SR_REAL = 1.0. At a 10% base rate, what share of passing ideas is now real?
5. Success: your step-1 predictions land within about 10% of the lab's averages, and you can say why a stronger edge raises the share of real passes although the size is unchanged.

1.4.9 Glossary · Check yourself · Go deeper

Signal-to-noise ratio — how much of an outcome's variation a signal explains relative to what it cannot; in daily returns even excellent signals explain well under 1% (

R^2 = \mathrm{IC}^2

Non-stationarity — the process generating the data changes over time (regimes, structural change, other traders adapting), so the past is an imperfect guide to the future.

Overfitting — fitting the noise in a sample as if it were signal: better in-sample, worse out of sample; it grows with the number of features and trials relative to the data.

Backtest-to-live decay — the drop in performance (e.g. Sharpe ratio) between a strategy's backtest and its live trading, from overfitting, missed costs and edge erosion together.

Base rate — how often an outcome occurs in the reference class a case belongs to; the prior you start from before weighing the specifics of the case.

Check yourself

1. A colleague's gradient-boosted model explains 6% of next-day stock returns in-sample and 0.2% out of sample. Which number is the red flag, and which might still be good news?

The 6%: in daily returns an in-sample fit that large is almost always fitted noise, roughly

K/T

. The 0.2% out of sample may be valuable: it is an IC of about 0.045, worth a Sharpe ratio near 0.7 before costs if traded daily. Judge it by its out-of-sample Sharpe ratio after costs.

2. A strategy has run live for two years with a Sharpe ratio of 0.6. Your manager asks whether it is "proven". What do you say?

No. On daily data the standard error after two years is roughly

1/\sqrt{2} \approx 0.71

, bigger than the estimate. Being 95% sure a Sharpe of 0.6 is above zero takes about

(1.96/0.6)^2 \approx 11

years. Judge it by its process.

3. You label each day with its 20-day forward return, run shuffled 5-fold cross-validation, and get 58% directional accuracy. What went wrong?

Leakage: neighbouring labels share 19 of their 20 days, so shuffled test folds contain near-copies of training samples. Use time-ordered validation with purging and an embargo (Session 6.6) and expect the accuracy to fall towards 50%.

4. A strategy made money for eight years, then lost money for the last twelve months. Kill it?

Not on that alone: in the lab, a strategy with a true Sharpe ratio of 0.79 loses money in 21% of years. Apply the kill criterion you wrote in advance, check whether the economic story still holds, and use a monitoring rule built to detect breaks (Session 9.9).

5. You think about 1 in 20 of your trading ideas is real. One passes a single, clean, 5-year backtest at 95% confidence. How likely is it to be real, if real ideas in your area have a Sharpe ratio of about 0.5?

About 30%: with a 5% base rate, a power of 20% and a size of 2.5%, Bayes' rule gives

0.05\,\text{power} / (0.05\,\text{power} + 0.95\,\text{size})

. A pass should move you from "probably not" to "worth paper trading", not to "deploy".

6. A day-trading course says its best students are in the top 1%, so with enough discipline you can be too. What is the base-rate answer?

In the Brazilian data, 97% of people who day traded for more than 300 days lost money and about 1% earned more than the minimum wage; the share who were profitable fell the more days people traded, and the persistent traders showed no learning. "I am disciplined" is exactly the inside view the base rate corrects.

🔧 Homework (~1 h before Session 1.5)

1. Draw your own fig 1.4.2, naming three bottom-right habits from your job and the session that replaces each.
2. In the lab, add 0.3 and 0.4 to SR_GRID and a 99% column, years_needed(x, z=2.576). Predict the years a Sharpe of 0.4 needs, then check.
3. Write an honest base rate for one trading idea of yours, naming the reference class. Compute the posterior with the lab's posterior_real() at a power of 20% and of 60%.
4. Read Wiecki et al. (2016) and Israel, Kelly & Moskowitz (2020), below, and make sure the lab runs on your machine: Session 1.5 — Set Up Your Quant Lab builds the workbench every later lab uses.

📚 Go deeper

Wiecki, T., Campbell, A., Lent, J. & Stauth, J. (2016). "All That Glitters Is Not Gold: Comparing Backtest and Out-of-Sample Performance on a Large Cohort of Trading Algorithms." The Journal of Investing 25(3), 69–80. doi:10.3905/joi.2016.25.3.069
Israel, R., Kelly, B. T. & Moskowitz, T. J. (2020). "Can Machines 'Learn' Finance?" Journal of Investment Management 18(2), 23–36. SSRN 3624052
Suhonen, A., Lennkh, M. & Perez, F. (2017). "Quantifying Backtest Overfitting in Alternative Beta Strategies." The Journal of Portfolio Management 43(2), 90–104. doi:10.3905/jpm.2017.43.2.090
Arnott, R., Harvey, C. R. & Markowitz, H. (2019). "A Backtesting Protocol in the Era of Machine Learning." The Journal of Financial Data Science 1(1), 64–74. doi:10.3905/jfds.2019.1.064
Carver, R. (2015). Systematic Trading: A Unique New Method for Designing Trading and Investing Systems. Harriman House. ISBN 978-0-85719-445-9
Tetlock, P. E. & Gardner, D. (2015). Superforecasting: The Art and Science of Prediction. Crown. ISBN 978-0-8041-3669-3

Sources for the numbers

Gu, S., Kelly, B. & Xiu, D. (2020). "Empirical Asset Pricing via Machine Learning." Review of Financial Studies 33(5), 2223–2273. doi:10.1093/rfs/hhaa009
Carver, R. (2025). "R squared and Sharpe Ratio." This Blog is Systematic, 1 November 2025. https://qoppac.blogspot.com/2025/11/r-squared-and-sharpe-ratio.html
Lo, A. W. (2002). "The Statistics of Sharpe Ratios." Financial Analysts Journal 58(4), 36–52. doi:10.2469/faj.v58.n4.2453
López de Prado, M. (2018). "The 10 Reasons Most Machine Learning Funds Fail." The Journal of Portfolio Management 44(6), 120–133. doi:10.3905/jpm.2018.44.6.120
Harvey, C. R. & Liu, Y. (2015). "Backtesting." The Journal of Portfolio Management 42(1), 13–28. doi:10.3905/jpm.2015.42.1.013
Chague, F., De-Losso, R. & Giovannetti, B. (2020). "Day Trading for a Living?" Working paper, version of 13 June 2020. SSRN 3423101
Barber, B. M., Lee, Y.-T., Liu, Y.-J. & Odean, T. (2014). "The Cross-Section of Speculator Skill: Evidence from Day Trading." Journal of Financial Markets 18, 1–24. doi:10.1016/j.finmar.2013.05.006
Barber, B. M., Lee, Y.-T., Liu, Y.-J., Odean, T. & Zhang, K. (2020). "Learning, Fast or Slow." Review of Asset Pricing Studies 10(1), 61–93. doi:10.1093/rapstu/raz006
Ioannidis, J. P. A. (2005). "Why Most Published Research Findings Are False." PLoS Medicine 2(8), e124. doi:10.1371/journal.pmed.0020124
Kahneman, D. & Lovallo, D. (1993). "Timid Choices and Bold Forecasts: A Cognitive Perspective on Risk Taking." Management Science 39(1), 17–31. doi:10.1287/mnsc.39.1.17

← Prev: 1.3 · Research Lifecycle Next: 1.5 · Lab Setup →

Educational material only — not investment, legal or tax advice. Backtests and simulations are hypothetical. Disclaimer · Text CC BY 4.0 · Code MIT.