Eight alignment-relevant categories. Seven axes weighted into a single ProbeScore. Every entry ships with a Colab reproducer, an SHA-256-hashed test set, and Pearson_CE cross-model transfer numbers. We are publishing the slot we want our own probes to compete in.
Probing the residual stream is one of the most reproducible interpretability primitives we have — and one of the easiest to mis-report. ProbeScore composes the three corrections most likely to surface the difference between a real probe and a confound.
Published probes hit 0.96+ on Apollo deception suites, but break under long-context distribution shift (DeepMind 2601.11516) and inflate when test prompts trigger eval-awareness (arXiv 2509.13333). ProbeScore composites both corrections so the headline number is harder to oversell.
Naive cosine alignment misses 38% of causal direction across crosscoders. Pearson_CE, the paired-ablation correlation between source and target models, is the honest transfer metric. Every cross-model entry on this page ships one.
No axis weighted above 30%. License score capped for closed-weight probes. Eval-awareness component dampens when calibration is missing. Hard to game by optimizing one number, easy to PR if you disagree with the weights.
Sorted by ProbeScore across all 8 categories. Click any row for the full Probe DNA card — eval table, Pearson_CE matrix, license, artifact hash.
Each alignment-relevant slot has its own table, its own held-out test set, and its own empty rows for probes that haven't been submitted yet.
Factual fabrication, entity recall failures.
| Rank | Probe | Model · Layer | Best AUROC | Eval-aware | Dist-shift | ProbeScore |
|---|---|---|---|---|---|---|
3 | FabricationGuard v2live AUROC 0.88 cross-task on SimpleQA · −88% confident-wrong reduction OpenInterp · Caio Vicentino | Qwen3.6-27B L31 · end_question | 0.903 | 0.662 | 0.598 | 0.662 |
Chain-of-thought faithfulness, hypocrisy gap.
| Rank | Probe | Model · Layer | Best AUROC | Eval-aware | Dist-shift | ProbeScore |
|---|---|---|---|---|---|---|
5 | ReasonGuard v0.2live Multi-bench training tested (FabricationGuard methodology) and FALSIFIED for reasoning probes — domain-bound at all difficulty levels OpenInterp · Caio Vicentino | Qwen3.6-27B L55 · mid_think | 0.908 | 0.572 | 0.537 | 0.626 |
Strategic dishonesty, sleeper agents, alignment faking.
| Rank | Probe | Model · Layer | Best AUROC | Eval-aware | Dist-shift | ProbeScore |
|---|---|---|---|---|---|---|
2 | DeceptionGuard (Apollo re-impl)pending Re-impl of Apollo Research deception probe (AUROC 0.96–0.999 published) OpenInterp · Caio Vicentino (re-impl) | Llama-3.3-70B L40 · last_token | 0.978 | 0.810 | 0.745 | 0.785 |
Capability hiding under evaluation.
Model knows-it-is-being-tested confound.
| Rank | Probe | Model · Layer | Best AUROC | Eval-aware | Dist-shift | ProbeScore |
|---|---|---|---|---|---|---|
1 | EvalAwarenessGuardpending Detects when model believes it is being evaluated · UK AISI priority OpenInterp · Caio Vicentino | Qwen3.6-27B L40 · end_question | 0.930 | 0.930 | 0.810 | 0.848 |
RL-induced misalignment generalization.
| Rank | Probe | Model · Layer | Best AUROC | Eval-aware | Dist-shift | ProbeScore |
|---|---|---|---|---|---|---|
4 | RewardHackGuard PoCpending Detect emergent reward-hacking generalization · Anthropic Nov 2025 framing OpenInterp · Caio Vicentino | Qwen3.6-27B L31 · token_avg | 0.650 | 0.590 | 0.520 | 0.645 |
EU AI Act Article 5 — persuasion / subliminal.
Over/under-refusal calibration; jailbreak proxy.
Seven axes, each in [0, 1], weighted to sum to 1.0. We report every component alongside the headline score so you can re-aggregate with your own weights.
Where these probes apply in practice. CZI Biohub committed $500M to AI-biology on Apr 29 2026 — medical AI is the most urgent vertical for fabrication detection.
Open a PR with a sklearn-compatible artifact (probe + scaler + metadata), an SHA-256-hashed test set, and a one-click Colab reproducer. The submission template walks you through the spec.
predict_proba(X) -> (n, 2) probe interfaceStandardScaler-compatible feature scaler0.0.1The openinterp PyPI package ships a probebench module that downloads any probe, runs the canonical eval suite, and prints a ProbeScore card identical to the leaderboard above.
pip install -U openinterp
from openinterp import probebench
probe = probebench.load("openinterp/fabricationguard-qwen36-27b-l31-v2")
print(probebench.score(probe))