Abstract
Functional Atomic Recomposition (FAR) composes higher-order cognitive functions on demand from atomic pieces obtained by Functional Atomic Decomposition (FAD). The cost that matters in FAR is selection: which atomic functions, out of a growing library, are relevant to a given recomposition. SWIFT attention mechanization answers that cost with attention-style relevance routing — score candidate functions against the request, activate the concentrated subset, and compose from it. This is the selective-activation principle that self-attention [1] made central and that FlashAttention [2] made cheap in memory and IO, applied to function selection rather than token mixing. The recomposition path rests on ordinary function-level programming [3], the modularity argument for higher-order functions [4], and functions as first-class values [5]. FlashAttention [2] is the benchmarked baseline the Transform step builds on and is measured against.
1. Introduction
FAD decomposes a task into atomic functions; FAR recomposes higher-order functions from them on demand. As the atomic library grows, the dominant cost is not executing a composition — it is finding the small set of atomic functions worth composing. Considering every candidate on every request makes recomposition scale with the library. SWIFT attention mechanization removes that coupling by scoring candidates for relevance and activating only the concentrated subset, so recomposition cost tracks the selected functions rather than the full library.
SWIFT organizes this into three routing behaviors — Dynamic Focused Attention (DFA), which concentrates compute on the high-relevance functions for a request; Hierarchical Memory Access (HMA), which tiers the atomic library so hot functions resolve first; and Parallelized Query Matching (PQM), which scores query–candidate relevance in parallel. Each is a routing stage, and each is placeable and swappable independently.
2. Related Work
Attention and efficiency. The relevance-weighting operation SWIFT routes on is self-attention [1]. FlashAttention [2] is the benchmarked technique for computing exact attention with far less memory traffic; SWIFT's Transform step builds on it, and any SWIFT speed claim is stated relative to it and settled against it on a shared workload.
Composition. Recomposing higher-order functions from atomic ones is function-level programming [3], the modularity case for higher-order functions [4], and functions as first-class values [5]. FAD/FAR applies that composition tradition to cognitive pipelines: atomic functions are the first-class values, and recomposition is higher-order composition over them.
3. SWIFT as attention-routed selective activation
The core mechanism is a conditional that holds whenever relevance concentrates: if attention-style scoring selects the small subset of atomic functions relevant to a recomposition, and if that selection is cheaper than considering every candidate, then recomposition cost scales with the selected subset rather than the full library. This is the selective-activation principle behind mixture-of-experts routing and the efficient-attention line [2], carried into function composition.
- DFA / HMA / PQM are the three routing stages: focus compute on high-impact functions, tier memory so hot functions resolve first, and parallelize query–candidate matching. Each names a stage of the route from request to composed function.
- Dense vs. sparse routing is the engineering axis: materialize every candidate, or store and score only the active entries. Sparse routing wins exactly when attention concentrates — a measurable property of the workload, and the variable that decides the outcome.
The scaling target follows from concentration. [ILLUSTRATIVE] Under attention that concentrates on
a top-k subset, the dense path scores candidates in O(n log n) and the sparse path in O(k log n)
for k ≪ n — the separation between the two paths is the concentration ratio k/n, which the
routing either achieves on a given workload or does not. FlashAttention [2] on a shared workload
is the head-to-head that fixes the constants.
4. IPO of the pipeline
- Input. A recomposition request plus a library of atomic functions.
- Process. Score candidate functions for relevance to the request (PQM); route to a dense or sparse path by how concentrated the relevant set is (DFA over an HMA-tiered library); execute; feed timing back to adjust future routing.
- Output. A composed higher-order function plus routing telemetry.
The feedback step closes the loop: measured routing cost adjusts the dense/sparse switch, so the pipeline tunes toward the concentration structure of the workload it actually sees.
Evidence & Scope
SWIFT attention mechanization is a design with one measurable pivot: whether attention-style scoring
concentrates relevance tightly enough that the sparse path (O(k log n)) beats the dense path
(O(n log n)) on a real recomposition workload. The O(n log n) / O(k log n) separation is
illustrative under concentrated attention; the concentration ratio k/n is the empirical property
that decides it, and FlashAttention [2] on a shared workload fixes the constants. The
recomposition path stands on function-level programming [3, 4, 5] and
the selective-activation line from self-attention [1] through FlashAttention [2].
FlashAttention is the baseline; attention concentration is the variable that decides the result.
References
- Ashish Vaswani et al. (2017). Attention Is All You Need. Advances in Neural Information Processing Systems (NeurIPS). arXiv:1706.03762.
- Tri Dao et al. (2022). FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awareness. Advances in Neural Information Processing Systems (NeurIPS). arXiv:2205.14135.
- John Backus (1978). Can Programming Be Liberated from the von Neumann Style? A Functional Style and Its Algebra of Programs. Communications of the ACM. [1977 ACM Turing Award Lecture]
- John Hughes (1989). Why Functional Programming Matters. The Computer Journal.
- Christopher Strachey (2000). Fundamental Concepts in Programming Languages. Higher-Order and Symbolic Computation. [Reprint of 1967 lecture notes]