Abstract

HOF cognition is the name this lineage gives to a single organizing idea: build task behavior by composing small, context-selected functions at runtime instead of running a fixed pipeline or retraining a monolith. The idea has three moving parts — decompose a capability into atomic functions, index them for approximate-nearest-neighbor retrieval [1], and recombine the retrieved subset by higher-order composition [2, 3, 4]. Two of those parts are established technology — scalable vector search and functional composition — which HOF cognition reuses by design. The contribution is the wiring: applying higher-order composition to the runtime orchestration of cognitive primitives, so the set of active functions changes with the task. This paper states the loop and gives one Input–Process–Output example [ILLUSTRATIVE]. Fine-tuning [5] and retrieval-augmented generation [6] are the adaptation baselines it is positioned against.

1. Introduction

Two adaptation strategies dominate practice. Fine-tuning [5, 7] specializes a model's weights but is subject to catastrophic forgetting [8, 9] and the cost of weight updates, reduced but not removed by low-rank methods [10, 11]. Retrieval- augmented generation [6] grounds output in retrieved passages but depends on retrieval quality and does not eliminate hallucination [12].

HOF cognition is a structural alternative orthogonal to both: keep a library of small functions and select and compose the relevant ones per request. The word "higher-order function" carries its ordinary programming-language meaning — a function that takes or returns functions [4] — and the case for building systems this way is the classic modularity argument for functional composition [2, 3]. The contribution is not the vocabulary but the wiring: context-driven retrieval [1] into that compositional structure, so the set of active functions changes with the task.

2. Related Work

Functional composition. Higher-order functions, currying, and composition are foundational programming-language concepts [4], argued as the basis of modular software by Backus [2] and Hughes [3]. HOF cognition inherits this vocabulary wholesale.

Vector retrieval. Selecting relevant primitives by similarity is approximate-nearest-neighbor search, a benchmarked systems technique [1]. Retrieval to condition generation is the RAG pattern [6].

Adaptation baselines and attention. Fine-tuning and parameter-efficient adaptation [5, 10, 11, 13] are the baselines a full evaluation measures against. The mechanism that ranks relevant functions is context-driven relevance weighting, anchored to transformer attention [14].

3. The Cognition Loop

A task is served by four composable steps:

  1. Decompose the capability into atomic, single-purpose functions (e.g., probability estimation, utility scoring, threshold comparison for a decision).
  2. Index and retrieve the context-relevant primitives by ANN search [1].
  3. Compose the retrieved primitives via higher-order composition [2, 3].
  4. Monitor the composed behavior and, on deviation, re-retrieve or recompose.

Steps 1–3 reuse established technique; step 4 is the control loop that makes the composition adaptive — the set of active functions is re-selected when the intermediate result diverges from the task.

4. Illustrative Worked Example

[ILLUSTRATIVE] In Input–Process–Output form, a reasoning-about-probabilities task:

INPUT: a query plus evidence
PROCESS: retrieve(pattern-match) ∘ retrieve(estimate-probability) ∘ retrieve(logical-inference)
 each retrieve(·) is an ANN lookup into the atomic-function index; monitor() re-runs
 retrieval if the intermediate result fails a validity check
OUTPUT: an inferred conclusion, or a request for more evidence

The example fixes the shape of the composition: each step is an ANN lookup into the atomic-function index, the steps compose left-to-right, and monitor() re-runs retrieval when an intermediate result fails a validity check.

Evidence & Scope

HOF cognition is a design with a testable core. Vector search [1] and functional composition [2, 3, 4] are the established components it composes; the contribution is their runtime wiring into a task-selected loop. The worked example is illustrative arithmetic on the composition's shape, not a benchmark. The claim it stands on is a mechanism: generalizing across tasks by recomposing reusable atomic functions rather than retraining weights — measured, on a shared task, against fine-tuning [5] and RAG [6]. Where "attention" appears outside transformer attention [14], it denotes context-driven relevance weighting.

References

  1. Jeff Johnson et al. (2019). Billion-Scale Similarity Search with GPUs. IEEE Transactions on Big Data. arXiv:1702.08734. [FAISS.]
  2. John Backus (1978). Can Programming Be Liberated from the von Neumann Style? A Functional Style and Its Algebra of Programs. Communications of the ACM. [1977 ACM Turing Award Lecture]
  3. John Hughes (1989). Why Functional Programming Matters. The Computer Journal.
  4. Christopher Strachey (2000). Fundamental Concepts in Programming Languages. Higher-Order and Symbolic Computation. [Reprint of 1967 lecture notes]
  5. Jeremy Howard & Sebastian Ruder (2018). Universal Language Model Fine-tuning for Text Classification. Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (ACL). arXiv:1801.06146. [ULMFiT]
  6. Patrick Lewis et al. (2020). Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks. Advances in Neural Information Processing Systems (NeurIPS). arXiv:2005.11401. [RAG]
  7. Jacob Devlin et al. (2019). BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding. Proceedings of NAACL-HLT. arXiv:1810.04805.
  8. Michael McCloskey & Neal J. Cohen (1989). Catastrophic Interference in Connectionist Networks: The Sequential Learning Problem. Psychology of Learning and Motivation.
  9. James Kirkpatrick et al. (2017). Overcoming Catastrophic Forgetting in Neural Networks. Proceedings of the National Academy of Sciences (PNAS). arXiv:1612.00796. [Elastic Weight Consolidation (EWC)]
  10. Edward J. Hu et al. (2022). LoRA: Low-Rank Adaptation of Large Language Models. International Conference on Learning Representations (ICLR). arXiv:2106.09685.
  11. Qingru Zhang et al. (2023). AdaLoRA: Adaptive Budget Allocation for Parameter-Efficient Fine-Tuning. International Conference on Learning Representations (ICLR). arXiv:2303.10512.
  12. Ziwei Ji et al. (2023). Survey of Hallucination in Natural Language Generation. ACM Computing Surveys. arXiv:2202.03629.
  13. Neil Houlsby et al. (2019). Parameter-Efficient Transfer Learning for NLP. Proceedings of the 36th International Conference on Machine Learning (ICML). arXiv:1902.00751.
  14. Ashish Vaswani et al. (2017). Attention Is All You Need. Advances in Neural Information Processing Systems (NeurIPS). arXiv:1706.03762.