Quantitative strategy research tackles two intertwined tasks: discovering a profitable rule and proving that the discovery is not merely a product of search luck. The latter requires admission gates—statistical criteria that must be satisfied before a conclusion is accepted. Yet the effectiveness and cost of such gates have never been rigorously tested.
We introduce an injected‑truth protocol: a known signal is randomly injected into real data, and a random‑admission control is constructed to adopt at the same rate as the gate. Only when the gate’s adoption rate exceeds that of the control does the gate convey information rather than simply raising a threshold.
Experiments on synthetic datasets and empirically calibrated panels reveal that, in the weak‑signal regime, gates substantially reduce false discoveries but also shrink actual adoption to 1%–7%. In the strong‑signal regime, gates add virtually no benefit. Most importantly, gates computed on absolute returns rather than excess returns silently reject every candidate, including true signals.
The findings urge researchers to consider signal strength, adoption rates, and the choice of return metric when designing admission criteria, lest overly stringent gates discard genuinely profitable strategies.
Review