dataphil lab

Working notes · models · results that didn't go the way I wanted

Where the models get tested before anyone trusts them

The product lives at dataphil.com. This is the other half: the forecasting work behind it, written up honestly — including, so far, mostly the parts that failed. Every result here comes with the raw data and code to recompute it.

Projects

The MLB shadow test

Not deployed

Jun–Aug 2026 · 10,928 out-of-sample predictions · Python, Monte Carlo, 7.8M pitches

The model forecast well and bet badly. Where it disagreed most with the market, it predicted 64.7% and delivered 54.0%.

A Monte Carlo simulator that prices MLB hitter props from pitch-level Statcast data. I wrote a pass/fail gate into the methodology document two days before the first prediction, then ran it against live sportsbook prices for 87 days without betting a dollar.

It failed. A head-to-head regression against the market price showed why: the model carries genuine information the closing line doesn't (β = 0.231, z = 2.20) but earns only about a quarter of the weight the edge formula was giving it. The decision rule was wrong, not the model — and neither was good enough to ship.

Read the writeup Try the explorer → Run it in Colab Data & code

Still accruing

Two more sub-models off the same simulator are collecting evidence and aren't ready to claim anything: a game-totals model, which runs the opposite direction from the hitter model on a sample far too small to trust yet, and a pitcher strikeout model whose variance is knowingly understated. Both are logging predictions daily under the same discipline. They'll show up here when there's something honest to say about them.