Abstract

Retrieval-augmented generation (RAG) [1] grounds a language model's output in retrieved context, reducing hallucination [2] but bounded by the quality of a retrieval step that, in most deployments, is a static nearest-neighbor lookup. A single fixed retrieval policy cannot be optimal across heterogeneous queries. This paper treats memory recall as a higher-order function — a retrieval strategy selected and composed at run time from atomic retrieval operations [3, 4, 5] rather than fixed at design time — so the retrieval stage adapts to context, task, and prior outcome. The design connects to parameter-efficient adaptation (LoRA [6], AdaLoRA [7]), to the catastrophic-forgetting problem that keeps knowledge external [8, 9], and to the psychophysics of a detection decision — Signal Detection Theory [10], sensitivity d′ and criterion β — that gives a recall accept/reject measurable structure. The open question it isolates is precise: whether a controller can select the task-relevant retrieval strategy accurately and cheaply enough that adaptive recall beats any fixed policy over a heterogeneous query distribution. RAG [1] is the baseline it is built to be measured against.

1. Introduction

A standard RAG pipeline [1] embeds a query, retrieves the top-k nearest passages by a fixed similarity metric, and conditions a generator [11, 12] on them. The retrieval policy — which index, which metric, which k, which re-ranker — is chosen once, offline. That policy cannot be optimal across heterogeneous queries: some want recency, some a broad recall net, some a single authoritative passage. A system that selects its retrieval strategy per query retrieves more relevant context precisely where a fixed policy is mismatched to the query. This paper makes that selection a first-class, composable operation rather than a design-time constant.

2. Related Work

RAG and its limits. RAG [1] is the baseline. Its documented failure mode is retrieval-dependence: faithfulness is bounded by retrieval quality, and hallucination survives imperfect grounding [2]. The intervention this paper names — adaptive retrieval-strategy selection — is a retrieval-quality intervention, so [2] is the yardstick.

Why keep memory external. Fine-tuning knowledge into weights risks catastrophic forgetting [8], mitigated but not solved by regularization such as EWC [9]. This is the standard argument for keeping knowledge in a retrievable store rather than in parameters, and it is the ground the present design stands on.

Adaptation as composable, budgeted structure. Parameter-efficient methods make adaptation cheap and modular: LoRA learns low-rank updates [6]; AdaLoRA [7] allocates the adaptation budget adaptively across modules. AdaLoRA is the closest published precedent for selecting the adaptation strategy at run time; the present design carries that adaptive-budget principle out of weight updates and into retrieval-strategy selection.

Composition. "Recall as a function that takes and returns other functions" is the standard definition of a higher-order function [3, 4, 5]. The contribution is applying run-time functional composition to memory recall.

Validating a recall. Whether a recalled item is accepted is a detection decision with a sensitivity (d′) and a criterion (β) — Signal Detection Theory [10], one applied psychophysical tool that gives the recall gate measurable structure and replaces intuition with two quantities.

3. Recall as a Higher-Order Function

Let a library of atomic retrieval operations exist (dense-kNN, recency-filtered, metadata-gated, multi-hop, re-ranked). A recall HOF is a function that, given the query, task descriptor, and a record of prior retrieval outcomes, composes a retrieval strategy from those atoms per request [3, 4]. Run-time composition retrieves more task-relevant context than any single fixed strategy averaged over a heterogeneous query distribution, for the same reason mixture-of-experts routing and AdaLoRA's adaptive budgeting [7] do: matching mechanism to input beats a fixed mechanism when the router is good.

The design isolates one question and states it sharply — whether that router exists, is cheap, and holds its win against its own overhead and error. That is the variable the measurement turns on.

4. Worked Example (Input–Process–Output)

Three queries against the same store, expressed as IPO transforms [4]:

This is [ILLUSTRATIVE] — it shows the shape of the design: different compositions for different queries. What the composer must get right is whether the chosen composition is the relevant one, and answer faithfulness [2] is the metric that scores it.

5. Validation

The validation loop, made precise, is retry and self-consistency over retrieval decisions gated by a detection criterion. Self-consistency over model outputs is a published technique [13]; the accept/reject gate is Signal Detection Theory [10] — the applied psychophysics of deciding, at a set criterion β, whether a recalled item clears the bar. Sensitivity and threshold are the two knobs, and both are quantities, not intuitions.

Evidence & Scope

This is a design with one isolated, empirical question. The worked example is illustrative arithmetic of compositional shape; the validation gate is grounded in Signal Detection Theory [10], d′ and criterion β; and what settles the design is a head-to-head between adaptive recall composition and a fixed-strategy RAG baseline [1] on a shared, heterogeneous query set scored by answer faithfulness [2], bounded by available compute. RAG is the baseline; the router that selects the composition is the variable that decides the outcome.

References

  1. Patrick Lewis et al. (2020). Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks. Advances in Neural Information Processing Systems (NeurIPS). arXiv:2005.11401. [RAG]
  2. Ziwei Ji et al. (2023). Survey of Hallucination in Natural Language Generation. ACM Computing Surveys. arXiv:2202.03629.
  3. John Backus (1978). Can Programming Be Liberated from the von Neumann Style? A Functional Style and Its Algebra of Programs. Communications of the ACM. [1977 ACM Turing Award Lecture]
  4. John Hughes (1989). Why Functional Programming Matters. The Computer Journal.
  5. Christopher Strachey (2000). Fundamental Concepts in Programming Languages. Higher-Order and Symbolic Computation. [Reprint of 1967 lecture notes]
  6. Edward J. Hu et al. (2022). LoRA: Low-Rank Adaptation of Large Language Models. International Conference on Learning Representations (ICLR). arXiv:2106.09685.
  7. Qingru Zhang et al. (2023). AdaLoRA: Adaptive Budget Allocation for Parameter-Efficient Fine-Tuning. International Conference on Learning Representations (ICLR). arXiv:2303.10512.
  8. Michael McCloskey & Neal J. Cohen (1989). Catastrophic Interference in Connectionist Networks: The Sequential Learning Problem. Psychology of Learning and Motivation.
  9. James Kirkpatrick et al. (2017). Overcoming Catastrophic Forgetting in Neural Networks. Proceedings of the National Academy of Sciences (PNAS). arXiv:1612.00796. [Elastic Weight Consolidation (EWC)]
  10. David M. Green & John A. Swets (1966). Signal Detection Theory and Psychophysics. Wiley.
  11. Ashish Vaswani et al. (2017). Attention Is All You Need. Advances in Neural Information Processing Systems (NeurIPS). arXiv:1706.03762.
  12. Tom B. Brown et al. (2020). Language Models are Few-Shot Learners. Advances in Neural Information Processing Systems (NeurIPS). arXiv:2005.14165. [GPT-3]
  13. Xuezhi Wang et al. (2023). Self-Consistency Improves Chain of Thought Reasoning in Language Models. International Conference on Learning Representations (ICLR). arXiv:2203.11171.