Compare selection rules against the same tape in the Model Comparison Lab

A "tape" is the generated sequence of events and prices a run is evaluated against. When two rules see the same tape, the comparison answers a genuinely useful question: if every rule had seen this exact sequence, which would have selected how much, and with what returns?

Without a shared tape, you are comparing rules and luck at the same time and cannot separate them. This is the single most common flaw in strategy claims, and it is a design property here rather than something you have to remember to control for.

The lab treats these as two distinct questions, and so should you. Selection is which events a rule bets at all — favorites only, underdogs only, mispriced by an edge filter, an odds threshold. Staking is how much it puts on each one — flat, a bankroll percentage, or a progression like Martingale.

Conflating them produces nonsense conclusions. A mediocre selection rule with an aggressive progression can post a spectacular headline return and a ruinous drawdown; a good selection rule staked flat can look dull and be far better. Read the two axes separately before you read them together.

Progression families — flat, Martingale, bankroll-percentage — are where staking gets interesting and dangerous. Run them on the same tape and look at the bankroll path and drawdown rather than the final number. Martingale in particular produces long stretches of smooth apparent success punctuated by a single catastrophic sequence, which is exactly why it survives as folklore.

Ruin risk is the metric that matters. A strategy that ends ahead in most runs but ends at zero in some is not a strategy that ends ahead, because you do not get to keep playing after zero.

The overlay breakdown compares multiple selection rules across the identical event set, showing what each would have selected and how those selections performed. It is the clearest view of whether a rule is finding something or simply betting more.

Watch for rules whose apparent edge comes from volume. A filter that selects nearly everything will track the tape average closely, which can look like consistency; a filter that selects rarely will be noisy regardless of quality. The overlay makes both visible.

The consistent price-band selection view isolates what a fixed odds range does to win rate and ROI, without chasing edges or streaks. This is useful precisely because it is boring: it shows the baseline behaviour of a band, which is the thing any cleverer rule has to beat.

A surprising number of strategies turn out to be a price-band bet wearing a hat. If your rule performs the same as flat betting the same band, the rule is not doing anything.

Lock the seed and rerun to confirm you get the identical result. Then change exactly one parameter — the progression, the filter, the band — and rerun. Attribution only works when one thing moved.

And keep the boundary in view. This is synthetic data with a known generating process. A rule that wins here has beaten a simulation whose assumptions you chose, which is a much weaker claim than beating a real market, and the lab is explicit that it does not accept wagers or claim real-world profitability.