Psychophysical Validation Frameworks for Cognitive Models

Abstract

LLM output validity is an open problem: models optimize for likelihood, not truth, and so hallucinate [1]. We frame an LLM output validator as a detector in the sense of Signal Detection Theory (SDT) [2]: the accept/reject decision is characterized by a sensitivity term (d′) and an adjustable decision criterion (β), and an adaptive retry loop fires when a candidate falls on the reject side of the criterion. SDT is one applied psychophysical tool here; Weber–Fechner scaling [3] and Stevens' power law [4] supply the log-magnitude foundation on which the decision threshold adapts, and the whole validate-and-retry architecture is expressed in cognitive DSL / IPO (Input–Process–Output) notation over the algebra of functional composition [5, 6, 7]. Measurement splits the result in two ◐[PARTIALLY-SUPPORTED]: the self-consistency validator is a genuine detector (d′ = 0.72, 95% CI [0.10, 1.35], lower bound > 0), while the reliability lift from adaptive retry is small and not statistically significant (Δ = +1.3%, McNemar p = 0.63). Both facts are reported plainly — a detector that resolves real signal and a retry loop that moves the needle little are the two things the data show.

1. Introduction

LLMs [8, 9] produce fluent text with no guarantee it is faithful to any external fact or internal logical constraint. That failure — hallucination — is documented and unsolved [1], and it is why output validation is needed at all. Two pressures compound it: overfitting to training patterns, which limits generalization, and contextual misalignment, where the model fails to prioritize the input that matters.

A validator makes an accept/reject decision under uncertainty. That is precisely the object Signal Detection Theory [2] describes, so we build the validator as a detector and measure it as one. The psychophysical toolkit — SDT's d′ and β, and Weber–Fechner just-noticeable-difference (JND) scaling for adaptive thresholds — is applied here as measurable structure.

2. Related Work

LLM validity and hallucination. Likelihood-trained models generate plausible but unsupported content, surveyed as an open problem [1]; retrieval grounding [10] mitigates but does not remove it.

Signal Detection Theory. SDT [2] is the mathematics of deciding "signal" versus "noise" under uncertainty. It separates sensitivity (d′, how distinguishable the two distributions are) from criterion (β, where the observer draws the accept/reject line) and summarizes the trade-off as a Receiver Operating Characteristic. It applies directly to an output validator, which must separate acceptable from unacceptable outputs and choose where to draw the line given the cost of each error type.

Weber–Fechner and Stevens. Fechner's logarithmic law [3] and Stevens' power law [4] relate stimulus magnitude to perceived magnitude on a compressive scale. They set the unit in which a threshold shift is meaningful: the JND. Adaptive thresholding is that principle applied to the accept/reject boundary — the decision criterion is moved in JND-scaled steps, not linear ones.

Functional composition. The validation, error-correction, and retry stages compose as higher-order functions over atomic functions, in the standard sense of function-level programming [5], the modularity argument for higher-order functions [6], and functions as first-class values [7].

3. Validation as Detection

An LLM output validator is a detector. Given a candidate output, it decides accept or reject; SDT [2] gives the two independent knobs that describe that decision:

• Sensitivity (d′) — how well the validator separates acceptable outputs from unacceptable ones. Improving d′ means building a genuinely better validator (e.g. cross-checking against retrieved evidence [10]); it cannot be obtained by moving the threshold.
• Decision criterion (β) — where the accept/reject boundary sits. Moving β trades false accepts (a flawed output passes) against false rejects (an acceptable output is discarded and retried). The right β depends on the relative cost of the two errors, which in agentic pipelines is asymmetric: a false accept propagates downstream, a false reject costs only a retry.

Adaptive thresholding is a precise operation: move β along the validator's ROC, in JND-scaled steps [3]. The frame also fixes what threshold adaptation cannot do — it cannot manufacture sensitivity the validator does not have.

4. The Validate-and-Retry Architecture (cognitive DSL / IPO)

The architecture composes a small set of stages, each written in cognitive DSL / IPO notation — Input, Process, Output — rather than any source language; composition is abstract per [5, 6].

• Logical-rule enforcement. Input: a candidate output plus the logical constraints it must satisfy (e.g. an implication p → q). Process: check the constraint. Output: pass/fail. This is the deterministic, cheap part of the gate — it catches contradictions that need no model call to detect.
• Error isolation. Input: a failing output and the stage that produced it. Process: localize the fault to the offending atomic step rather than the whole chain. Output: the isolated faulty step. Isolation prevents a single bad step from forcing the whole workflow to restart.
• Adaptive retry. Input: a rejected output and a finite retry budget. Process: refine the input (or adjust β) and re-invoke, decrementing the budget. Output: an accepted output, or an explicit failure marker when the budget reaches zero. The loop terminates by construction.
• Compositional validation. Input: a multi-step reasoning chain. Process: validate at each composition boundary, not only at the end. Output: a chain whose every intermediate step has passed its gate. This is the functional-composition guarantee applied to control flow [6].
• State immutability. Input: intermediate states. Process: keep them immutable so a retry re-derives rather than mutates. Output: reproducible, debuggable traces.

For the retry budget, one ○[ILLUSTRATIVE] identity: if a validator's per-step accept probability is a and the retry budget is r, the probability that at least one of r independent attempts is accepted is 1 − (1 − a)^r. This is arithmetic over independent Bernoulli trials under an assumed a; the independence assumption is the term to watch, since retries re-sample the same model on a similar prompt.

5. Measurement: detection is real, the reliability lift is not

The SDT frame is tested directly, and the result is ✓[MEASURED] and ◐[PARTIALLY-SUPPORTED] — it splits in two.

Detection is real. A self-consistency validator achieves a measured sensitivity of d′ = 0.72 (95% CI [0.10, 1.35], lower bound > 0), with criterion β = 1.95 — above-chance discrimination between acceptable and unacceptable outputs (N = 156, first-shot-only, Hautus log-linear correction, equal-variance Gaussian). A pure format validator shows d′ ≈ 0 (CI covers zero): it does not discriminate over value errors. The detector resolves signal the format check cannot.

The reliability lift is not. End-to-end task accuracy moves from 20.5% to 21.8%, a Δ = +1.3% that is not statistically significant (McNemar p = 0.63). Retry helps little here because the model's errors are largely systematic — a re-sample reproduces the same wrong answer — so a detector that sees an error still cannot fix it by asking again.

The reading is clean: the SDT method is validated as a genuine detector, which is the substantive contribution, while retry-driven reliability is a modest, non-significant effect. A null is evidence; both halves are reported as measured.

Evidence & Scope

SDT-as-detector is carried by measurement: the self-consistency validator resolves real signal (d′ = 0.72 [0.10, 1.35], β = 1.95, N = 156), while a pure format validator does not (d′ ≈ 0). The retry-driven reliability gain is a small, non-significant effect (Δ = +1.3%, McNemar p = 0.63), bounded by systematic model error rather than by the frame. SDT [2] describes the accept/reject trade-off every validator faces; sensitivity is established by measuring a specific validator on a specific task, and it is established here for the self-consistency detector. Weber–Fechner [3] and Stevens [4] set the log-magnitude scale on which the decision threshold adapts. A bounded retry loop re-samples the same underlying model, so where the model cannot produce an acceptable output the loop converges on the explicit failure marker. "HOF cognitive psychophysical validation" is a term of this corpus.

References

1. Ziwei Ji et al. (2023). Survey of Hallucination in Natural Language Generation. ACM Computing Surveys. arXiv:2202.03629.
2. David M. Green & John A. Swets (1966). Signal Detection Theory and Psychophysics. Wiley.
3. Gustav Theodor Fechner (1860). Elemente der Psychophysik. Breitkopf und Härtel.
4. S. S. Stevens (1957). On the Psychophysical Law. Psychological Review.
5. John Backus (1978). Can Programming Be Liberated from the von Neumann Style? A Functional Style and Its Algebra of Programs. Communications of the ACM. [1977 ACM Turing Award Lecture]
6. John Hughes (1989). Why Functional Programming Matters. The Computer Journal.
7. Christopher Strachey (2000). Fundamental Concepts in Programming Languages. Higher-Order and Symbolic Computation. [Reprint of 1967 lecture notes]
8. Ashish Vaswani et al. (2017). Attention Is All You Need. Advances in Neural Information Processing Systems (NeurIPS). arXiv:1706.03762.
9. Tom B. Brown et al. (2020). Language Models are Few-Shot Learners. Advances in Neural Information Processing Systems (NeurIPS). arXiv:2005.14165. [GPT-3]
10. Patrick Lewis et al. (2020). Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks. Advances in Neural Information Processing Systems (NeurIPS). arXiv:2005.11401. [RAG]