Built Not to Lie

The default failure mode of quantitative trading tooling is looking better than it is. Warren's third design commitment is a layer whose entire purpose is to refuse to make claims it cannot defend.

Every conclusion Warren draws, "this backtest is profitable," "this regime is bullish," "this Sharpe is real," passes through validity checks before it reaches you. Each check targets a specific, well-understood way a naive number misleads. If a check cannot confirm a conclusion, it does not shrug and pass the number through: it downgrades or withholds the verdict and says exactly why. The mechanics live in The Validity Layer; this page is the philosophy and the trust contract.

Verdicts, not scores

An earlier version of the product surfaced a single confidence number, and first-party analysis found it was non-discriminative: over a large sample of decided paper signals it barely moved off one value and had essentially zero correlation with realized outcomes. You cannot calibrate a probability on a signal that does not vary.

The lesson is encoded everywhere: do not replace "trust this 0.30" with "trust this 0.62." Decisions come from explicit hard rules producing three-way verdicts, ok, caution, or abstain, never from a threshold on one opaque score. Abstaining is a correct outcome, not an error.

The trust hierarchy

Warren still reports several "how much to trust this" figures, but only two of them are decision gates; the rest are transparency for display. Confusing the two is the exact anti-pattern the design set out to kill.

The gates decide; the display-only figures explain. A display-only number is never allowed to become a filter, a sort key, or a standalone probability, because that is exactly how a non-discriminative score quietly re-enters through the back door.

Four invariants

Every check in the layer obeys the same four rules:

• Deterministic and label-free. Verdicts are pure functions of inputs. No fitted model, fully testable, and immune to contaminated feedstock.
• Additive. A check annotates a result; it never rewrites the underlying numbers.
• Fail toward abstain. A check that could not run degrades the verdict; it never gets the benefit of the doubt. A money-touching "fail-open" is treated as the cardinal bug.
• Grounded, not self-reported. Anything checkable against data is checked in code, never by asking the model to critique itself, which rationalizes rather than verifies.