Abstract

Long-horizon agent tasks — end-to-end software delivery, autonomous MLOps — require carrying context across many steps, which fixed-window transformer models [1] do not natively do. This paper frames memory recall as a higher-order function: rather than a single static retrieval step, recall is composed at run time from atomic retrieval operations [2, 3, 4], selecting only the task-relevant subset of prior state to reload. The framing stands on the two constraints it addresses — the catastrophic-forgetting argument for keeping knowledge external and retrievable [5, 6], and the retrieval-quality bound on grounded generation [7, 8] — and on the efficient-attention result that bounds the window it feeds, FlashAttention [9]. Memories are weighted by salience and admitted through a detection gate; Signal Detection Theory [10] gives that gate measurable structure, and the Weber–Fechner just-noticeable-difference sets the threshold at which two candidate memories are distinguishable at all. The architecture centers on the per-step recall policy — what to reload and what to drop.

1. Introduction

An agent overseeing a multi-stage pipeline must relate a decision made at step 1 to a validation at step 40. Transformer context windows [1] are finite, and naively concatenating all prior state is both expensive and noisy. Two design responses exist: enlarge or compress the window, or keep state in an external store and retrieve the relevant fragment on demand. This paper concerns the second, and its thesis is sharp: the retrieval policy for long-horizon state should be selected per step, not fixed, because relevance is step-dependent. What is worth recalling at a deployment gate is not what was worth recalling during early design, and a fixed retrieval step cannot track that shift.

The mechanism is composition. Recall is not one operation but a family of atomic retrieval operations — by recency, by stage, by dependency, by semantic similarity — assembled per step into the strategy that step calls for. That assembly is a higher-order function: a function that takes the step's descriptor and returns the retrieval function to run.

2. Related Work

Why memory should be external. Writing long-horizon knowledge into weights invites catastrophic forgetting [5], only mitigated by regularization such as EWC [6]. This is the standard justification for a retrievable external memory rather than parametric memory — the foundation the present framing stands on.

Retrieval grounding and its limits. Retrieval-augmented generation [7] is the mechanism by which retrieved state conditions generation; retrieval quality bounds faithfulness, and hallucination persists under imperfect grounding [8]. A better recall policy is, concretely, an intervention on this bound.

Efficient attention. FlashAttention [9] is IO-aware exact attention with reduced memory traffic — the benchmarked result that makes the working window cheaper to fill. It makes attention over what is recalled cheaper; it does not by itself extend the semantic horizon of memory, which is the retrieval problem this paper addresses. The two compose: recall selects the fragment, FlashAttention attends over it efficiently.

Composition. Recall that takes context and returns a retrieval function is a higher-order function in the ordinary sense [2, 3, 4], and the retrieval atoms it composes are atomic functions. The contribution is applying that composition to per-step memory selection.

Salience and detection. Memories are weighted by salience and admitted through a threshold, which is a detection problem. Signal Detection Theory [10] gives it structure: accepting or rejecting a recalled item separates into sensitivity (d′) and criterion (β), and the recall/precision trade-off is a criterion shift along an ROC curve. The Weber–Fechner just-noticeable-difference fixes the resolution below which two candidate memories are indistinguishable and need not be scored apart. These are the applied psychophysics of the recall gate.

3. The Framing: Recall as Composition

Model long-horizon memory as a store of prior state plus a library of atomic retrieval operations (by-recency, by-stage, by-dependency, by-semantic-similarity). A recall HOF takes the current step's descriptor and returns a composed retrieval strategy over those atoms [3], reloading only the relevant fragment. In Input–Process–Output terms:

The property this buys is a bounded working set: you pay for what you retrieve, not for the whole history. Selection is where the leverage sits — a recall policy that admits the wrong fragment starves the step, and one that admits too much reintroduces the noise the window was meant to escape. Salience weighting and the detection gate are what keep the admitted set both small and sufficient.

4. Worked Example (Input–Process–Output)

An agent at the deployment-validation step of a software pipeline:

This is [ILLUSTRATIVE] — it shows the intended behavior of the composition on a concrete step. The per-step recall policy is what selective reload turns on: it preserves the context needed to make the correct call while dropping the 40-step history.

5. Where Validation Is Grounded

Recall decides accept or reject on each candidate memory, and that decision is a detection problem. Signal Detection Theory [10] gives it measurable structure: the recall/precision trade-off is a criterion (β) shift along an ROC curve, and recall quality separates into sensitivity (d′) versus threshold. Salience weighting sets what enters the decision; the Weber–Fechner just-noticeable- difference sets the resolution at which two candidates are worth distinguishing. This is the measurable core of the recall gate — the point where the framing meets a number.

Evidence & Scope

The recall HOF is a design with an isolated, testable variable: the per-step retrieval policy. The external-memory foundation is the catastrophic-forgetting argument [5, 6]; the payoff is an intervention on the retrieval bound that limits grounded generation [7, 8]; the working window it feeds is made cheap to attend over by FlashAttention [9]; and the accept/reject gate is grounded in Signal Detection Theory [10] with thresholds set at the just-noticeable-difference. The worked example is illustrative of the composition's behavior; the recall gate's contribution is the per-step selective-reload policy — the accept/reject decision grounded in Signal Detection Theory [10] at the just-noticeable-difference.

References

  1. Ashish Vaswani et al. (2017). Attention Is All You Need. Advances in Neural Information Processing Systems (NeurIPS). arXiv:1706.03762.
  2. John Backus (1978). Can Programming Be Liberated from the von Neumann Style? A Functional Style and Its Algebra of Programs. Communications of the ACM. [1977 ACM Turing Award Lecture]
  3. John Hughes (1989). Why Functional Programming Matters. The Computer Journal.
  4. Christopher Strachey (2000). Fundamental Concepts in Programming Languages. Higher-Order and Symbolic Computation. [Reprint of 1967 lecture notes]
  5. Michael McCloskey & Neal J. Cohen (1989). Catastrophic Interference in Connectionist Networks: The Sequential Learning Problem. Psychology of Learning and Motivation.
  6. James Kirkpatrick et al. (2017). Overcoming Catastrophic Forgetting in Neural Networks. Proceedings of the National Academy of Sciences (PNAS). arXiv:1612.00796. [Elastic Weight Consolidation (EWC)]
  7. Patrick Lewis et al. (2020). Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks. Advances in Neural Information Processing Systems (NeurIPS). arXiv:2005.11401. [RAG]
  8. Ziwei Ji et al. (2023). Survey of Hallucination in Natural Language Generation. ACM Computing Surveys. arXiv:2202.03629.
  9. Tri Dao et al. (2022). FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awareness. Advances in Neural Information Processing Systems (NeurIPS). arXiv:2205.14135.
  10. David M. Green & John A. Swets (1966). Signal Detection Theory and Psychophysics. Wiley.