The launch gate
Before an algorithm can be listed, it has to survive a battery of tests designed to tell a real edge from a curve fit.
This matters more than it sounds. Anyone with a fast backtest and a row of sliders can produce a strategy with a beautiful historical record by searching until the noise lines up. A headline Sharpe ratio measures how hard someone searched, not how good the strategy is.
Everything below is recomputed on our side, from the strategy itself, against our bars and our fee model. Nothing is taken on the submitter’s word — which matters most for strategies deployed through the API from somebody’s own backtester, where this gate is the only thing standing between a spreadsheet and a listing.
Four things that cannot be traded away
Section titled “Four things that cannot be traded away”These are eligibility, not quality. Failing any one of them refuses a strategy outright, however good the rest of it looks.
Significance. We take the strategy’s own signal and slide it along the price series so it is pointing at the wrong moments, then re-run it — five hundred times. That produces the distribution of results this exact strategy gets with no timing information at all, and the real run has to be out in its tail.
Sliding rather than shuffling is the whole trick. A shuffled signal flips every bar, so it trades constantly, pays enormous costs and loses badly — which would make almost anything look brilliant by comparison. A slid signal keeps every run, every holding period and therefore every fee. The one thing destroyed is whether it was pointing at the right moment, which is the only thing being tested.
A holdout the author never sees. A recent window withheld from the builder entirely. Passing it is not proof; failing it is disqualifying.
Tradeability. Enough rebalances, and enough movement to be worth taking a position in. People trade these algos — an instrument that barely moves is a fee with a chart attached.
Ruin. Not the worst drawdown in the backtest, which is by construction the unluckiest stretch in one sample and the least reproducible number in the whole thing. We resample the strategy’s own returns in blocks a few thousand times and read the 95th percentile: how bad it gets on an unkind draw of the same process. Past the point where an account does not come back, it is not listed.
That same resampled figure is what sizes your exposure when you do hold it.
Six things that are scored
Section titled “Six things that are scored”Everything else is a dimension, scored from 0 to 1 and combined into a single number a strategy has to clear.
Several of them read the out-of-fold figure rather than the full-sample one, which means this: the return series is cut into slices, and only the later part of each slice counts, with a gap — an embargo — before it, so a rolling indicator cannot leak across the boundary. A full-sample Sharpe is a statement about how hard somebody searched.
We are careful not to call that a walk-forward, because it is not one. A walk-forward refits the strategy on each training window and scores the next; this scores one configuration on slices it was not chosen on. That is a weaker claim, and naming it for the stronger one would be a way of overstating the evidence to ourselves.
| What it measures | |
|---|---|
| Edge | The out-of-fold Sharpe at the 5th percentile of a resampling — what it is worth on a bad draw, not on the one draw history handed us |
| Robustness | How much of the performance survives nudging every parameter and lookback by ±10% |
| Consistency | How many of the out-of-sample windows are positive, rather than one brilliant period carrying five poor ones |
| Cost headroom | The multiple of our fee model at which the edge reaches zero |
| Discipline | The deflated Sharpe: given how many configurations were tried, how surprising is the best one |
| Originality | How correlated it is with the closest algo already listed |
The score is a geometric mean, and that is not a detail. Under an ordinary weighted average, the cheapest way to raise a score is to pour everything into whichever dimension is easiest — which is always the one least connected to whether the strategy works. A geometric mean has no such trade: falling to a tenth on any one dimension costs a tenth of the whole score, whatever the other five say. Being outstanding at five things does not buy being hopeless at the sixth.
Every threshold and every curve is published, on purpose. A rule you cannot read is one you can only find by resubmitting, and resubmitting is more search.
Why a score, and not a list of pass marks
Section titled “Why a score, and not a list of pass marks”The gate used to be eight thresholds joined by AND. It failed in both directions at once.
It was too strict, in a way no single threshold revealed. An AND is only as permissive as its harshest term, and with three years of history the deflated Sharpe threshold alone demanded an out-of-fold Sharpe near 2.5. Every other test was decorative.
It was also too lax where it did bite. Requiring four of six out-of-sample windows to be positive passes a strategy with no edge whatsoever about a third of the time, and rejects a genuinely good one almost half the time — because six five-month windows are six coin flips. A hard line drawn on a measurement that noisy is a lottery with a threshold painted on.
Three years is not much data. Our estimate of a Sharpe ratio has a standard error around 0.7, so a strategy at 1.09 and one at 1.11 are the same strategy. Scoring across six dimensions averages out some of that noise; drawing a cliff in the middle of it does not.
How hard it actually is
Section titled “How hard it actually is”Tested adversarially against pure noise — data with no edge in it at all, where every pass is by definition a false positive — a search that produced 296 strategies with an in-sample Sharpe above 1.0, the best reaching 2.54, had none pass the gate. With zero passes over 296, the 95% upper bound on the true rate is 1.0%.
What caught them is the part worth reading:
| share of the 296 | |
|---|---|
| Significance | 98% |
| Holdout | 95% |
| Edge | 63% |
| Discipline | 8% |
| Ruin | 3% |
| Robustness | 2% |
The old gate caught 233 of 234 with the deflated Sharpe alone, which made it a one-test gate in all but name — and that test’s input is the number of configurations someone says they tried, which a fresh wallet resets. It now catches 8%, and the two tests that do the work are ones a creator cannot reset: a strategy’s own null distribution, and a window they have never seen.
Their median score was 0.30. The highest any of them reached was 0.93, above the passing mark — so the score alone would have let a few through. That is what the vetoes are for, and why they are not scored.
One more thing that run already assumes: it prices every candidate as having searched the minimum we will ever credit, which is what someone gets if they report nothing and start fresh every time. It is not a best case. Strip even that floor away — a state the API does not allow — and 6 of the 296 get through. Under the old arrangement, the same trick defeated the gate outright, because the one test doing nearly all the work read that number directly.
Passing the gate is not a prediction
Section titled “Passing the gate is not a prediction”It means a strategy is not obviously a fit. It is not a claim that it will make money. An algo that passes can still lose, and the track record that follows is what you should actually be reading.