How it was tested: the record does not state separate training and test periods; out-of-sample design: holdout.
Horizon: 10 sessions (label and headline Strength); up20/exp20 also carried at 20 sessions. Coverage: ~1,700-name panel (S&P 500 + Nifty + curated watchlists); not the 10k+-name scanner universe.
Stage: Production. Version v1.1; the hub record is as of 6 Nov 2026.
The verdict
FIT FOR LIMITED USE: the holdout AUC and calibration are genuine, but measured before the production refit folds the holdout back in, so live-model discrimination is not confirmed after that refit.
Development summary: Per-direction logistic models separate breaks from holds with holdout AUC 0.72 (highs) / 0.74 (lows) and well-behaved calibration terciles — real, independently-measurable discrimination, not just a descriptive base rate.
What the review allows
Strength/"Break 10d" as a directionally meaningful, better-than-chance probability (holdout AUC 0.72-0.74).
As M-011's quality_dip veto, exactly as built.
The measured base-rate asymmetry (momentum-approach ~85% vs true-test ~coin-flip) as a qualitative read even without the model score.
What the review rules out
Treat Strength as calibrated on the CURRENT production model — calibration was measured on a holdout that is no longer held out (it is folded into the refit).
Rely on Strength alone without Approach — most of its discrimination is one feature (distance to level).
Extend the veto logic beyond M-011 without re-checking region coverage for the target use.
What it predicts
Item
Detail
Question it answers
When price re-enters ±1 ATR of a prior 100d/250d/all-time high or low, what is the probability that level breaks (bullish at a high, bearish at a low) within the next 10 sessions?
Method
Per-direction logistic
How it works
Per-direction logistic regression (11 features, StandardScaler) fit on the full retest history; EB-pooled base rates by duration x direction x kind x regime, blended with each symbol's own record
Inputs
Distance to level (in ATRs), Approach side (from above / below), Prior touches of the level, Volatility vs 60 days ago, Market regime (risk-on / off)
Output
Strength (per-direction model probability the level breaks within 10 sessions), pool_rate, up10/up20/exp20 measured drift, region
Horizon
10 sessions (label and headline Strength); up20/exp20 also carried at 20 sessions
Coverage
~1,700-name panel (S&P 500 + Nifty + curated watchlists); not the 10k+-name scanner universe
Markets
US, India
Model type
Model · ML/Stat
Stage
Production
Version
v1.1, first recorded 3 Sep 2026
Validation evidence
Training data: not stated. Test data: not stated. Out-of-sample design: holdout.
Headline results from the model's development record, as the hub publishes them.
Distance to level (ATRs): +0.88 highs / −0.89 lows
next-largest coefficient ≤0.12
the model is overwhelmingly a distance-to-level score
The hub's standard metrics for this model class (only those the record measured).
Metric
Sample / window
Compared with
Value
Range (CI)
AUC (headline)
548d holdout · highs (lows 0.738)
0.5
0.722
n/a
Precision at k
548d holdout · top tercile, highs
62.2
87.60%
n/a
Brier score
548d holdout · highs · v1.1 fit
n/a
0.2004
n/a
What the validation review found
Is the design sound? Sound: per-direction models fix a real anti-calibration defect in a prior shared-model design (documented in the engine's own docstring), and separating the fitted probability (Strength) from the descriptive pool/drift rates is the right structure.
Data quality. Large event base (2.45M) with a sensible frozen-level definition (no lookahead in level choice). Region bucketing depends on universe coverage; not independently audited here.
Backtest. The 18-month holdout is a legitimate out-of-time test, but its result describes a model that is then discarded in favour of a full-data refit for production — see M-1.
Monitoring. None beyond the per-run printed log; no drift/PSI monitoring on the 11 features or on Strength's realized hit rate is evidenced. Addendum 2026-09-24: each run now persists the per-direction weights, scaler and holdout AUC/Brier; PSI and a realised hit-rate loop are still absent.
Limitations
The model is overwhelmingly a distance-to-level score (coefficient +0.88/−0.89 vs ≤0.12 for every other feature) — most of the AUC is one variable doing the work.
Coefficients are refit on all data (including the holdout) every run, so the exact weights scoring today's rows were never themselves out-of-time tested. Since 2026-09-24 each production fit is persisted and versioned, so the weights that scored any day can be audited and later scored on events after that date.
Base rates differ sharply by approach (momentum approaches break ~85% of the time almost mechanically; true tests are closer to a coin flip) — reading Strength without Approach can mislead.
Region-aware drift buckets fall back to global below 100 events; thin regions may carry a poorer-fitting base rate.
Questions
Is the High/Low Retest Strength validated?
Its validation review dated 3 Sep 2026 rated it FIT FOR LIMITED USE: the holdout AUC and calibration are genuine, but measured before the production refit folds the holdout back in, so live-model discrimination is not confirmed after that refit.
What does the High/Low Retest Strength predict?
It answers: When price re-enters ±1 ATR of a prior 100d/250d/all-time high or low, what is the probability that level breaks (bullish at a high, bearish at a low) within the next 10 sessions? Output: Strength (per-direction model probability the level breaks within 10 sessions), pool_rate, up10/up20/exp20 measured drift, region. Horizon: 10 sessions (label and headline Strength); up20/exp20 also carried at 20 sessions. Coverage: ~1,700-name panel (S&P 500 + Nifty + curated watchlists); not the 10k+-name scanner universe.
How was the High/Low Retest Strength tested?
Training data not stated, test data not stated, out-of-sample design: holdout. Holdout AUC, highs / lows: 0.722 / 0.738; Base break rate, highs / lows: 62.2% / 52.9%; Momentum-approach vs true-test break rate: ~85% vs ~coin flip.