RAAFT vs LoRA and Full Fine-Tuning for Reasoning

Abstract

Reasoning generalization over time — keeping a model useful as tasks and distributions drift — turns on how the model adapts. Three parameter-efficient fine-tuning (PEFT) strategies span the design space: full fine-tuning updates all weights; LoRA [1] adapts at a fixed low rank; adaptive-rank methods led by AdaLoRA [2] vary the rank per layer. This paper compares them along the axes that govern drift — reasoning capacity, adaptivity, adaptation granularity, forgetting exposure, and cost — and places RAAFT/HORA, a context-conditioned and token-composed variant, at the adaptive end of that spectrum. Where we measured, we report the measurement: on RoBERTa-base RTE [3], three seeds per arm, DoRA 0.743 > LoRA 0.727 > full fine-tuning 0.718 > AdaLoRA 0.651 ✓[MEASURED]. The comparison isolates one sharp question: whether a context controller can allocate rank per task at low enough overhead that adaptive adaptation matches or beats the strongest tuned baseline at equal parameter budget. AdaLoRA [2] is the state-of-the-art adaptive baseline the question is measured against.

1. Introduction

A model's long-run usefulness depends on generalizing as tasks and data distributions drift. LoRA [1] adapts at a fixed rank; full fine-tuning updates all weights and is the most exposed to catastrophic interference [4, 5]; AdaLoRA [2] varies rank adaptively. RAAFT/HORA (see the companion RAAFT paper) extends the adaptive direction: it conditions rank on runtime context and composes updates from atomic tokens. The governing question is which adaptation strategy holds up as the world moves. This paper compares the strategies structurally along the axes that decide drift behavior, names the single empirical question that separates them, and reports the RTE arm we have measured.

2. Related Work

PEFT baselines. LoRA [1] (fixed-rank), AdaLoRA [2] (adaptive-rank, the state-of-the-art comparison point for any adaptive method), and adapters [6]. A claim that context-conditioned adaptation beats the state of the art is a claim against AdaLoRA, not against LoRA and full fine-tuning alone.

Forgetting and continual adaptation. Sequential-learning interference [4] and its partial mitigation [5] are the phenomena behind the "stability over time" axis. A frozen base limits exposure; none of the methods compared here claims a dedicated forgetting remedy beyond that.

Reasoning and evaluation. SuperGLUE [3] is the benchmark family this comparison draws on. Self-consistency decoding [7] is a separately published technique for improving reasoning reliability; it is orthogonal to the choice of PEFT method and locates "reasoning generalization" in the established literature.

3. Qualitative Comparison

The axes below govern how an adaptation strategy behaves as tasks drift. Entries are directional expectations; the RTE arm in §5 is the one axis carrying a measured number.

🔊 This table lays out Axis, Full fine-tuning, LoRA (fixed rank) [1], Adaptive / context-conditioned rank [cf. @zhang2023adalora] across 5 rows.

The right-hand column's advantages are conditional on one thing: a context controller that allocates rank correctly and cheaply. That condition is the substance of the comparison. A fixed-rank baseline is not quadratic: a LoRA update of rank r over a dimension-n layer costs O(r·n) with r ≪ n [1], and any complexity a context-conditioned method claims is a design target to be derived and measured against that O(r·n) floor, not asserted.

4. Illustrative Cost Arithmetic

The efficiency argument for adaptive rank is arithmetic under an explicit assumption. ○[ILLUSTRATIVE] Suppose a task tolerates an average effective rank r̄ that is a fraction φ of a fixed worst-case rank r_fixed. The adaptation parameter count then scales with φ·r_fixed instead of r_fixed — a saving of (1 − φ); at φ = 0.5, a 50% parameter reduction. The empirical content sits entirely in whether a controller can choose r̄ ≈ φ·r_fixed without costing accuracy — the same quantity AdaLoRA measures directly [2].

The reasoning-time analogue names a concrete baseline. ○[ILLUSTRATIVE] Let Baseline B be a monolithic sequential reasoner that evaluates k candidate continuations of O(n) work each, for O(k·n) total. A selective, context-gated approach that evaluates only m ≪ k candidates scales as O(m·n) — a saving of (1 − m/k) under an assumed selectivity m/k. The empirical content is whether a controller picks the right m candidates.

5. Measured Result

On RoBERTa-base RTE [3], three seeds per arm, mean (range): ✓[MEASURED] DoRA 0.743 (0.729–0.755), LoRA 0.727 (0.704–0.751), full fine-tuning 0.718 (0.704–0.733), AdaLoRA 0.651 (0.635–0.682). On this arm the decomposition-based DoRA leads, fixed-rank LoRA and full fine-tuning follow, and the adaptive-rank baseline (AdaLoRA) sits lowest. One benchmark arm does not settle a five-axis comparison, and this one does not flatter the adaptive hypothesis — which is why it is the load-bearing datum. The bar a context-conditioned method has to clear is the strongest tuned baseline at equal parameter budget; on RTE that is DoRA at 0.743.

Evidence & Scope

This is a structural comparison with one measured arm. The qualitative axes are directional expectations; the cost arithmetic is illustrative under stated assumptions; the RTE result is measured on RoBERTa-base across three seeds per arm. RAAFT/HORA enters as a design on the adaptive-rank spectrum, and its advantage is conditional on a context controller that allocates rank correctly at low overhead — the one quantity every adaptive method, AdaLoRA included, earns empirically. The comparison point is AdaLoRA [2], and the strongest tuned baseline at equal parameter budget is the bar any context-conditioned method clears to earn its place.

References

1. Edward J. Hu et al. (2022). LoRA: Low-Rank Adaptation of Large Language Models. International Conference on Learning Representations (ICLR). arXiv:2106.09685.
2. Qingru Zhang et al. (2023). AdaLoRA: Adaptive Budget Allocation for Parameter-Efficient Fine-Tuning. International Conference on Learning Representations (ICLR). arXiv:2303.10512.
3. Alex Wang et al. (2019). SuperGLUE: A Stickier Benchmark for General-Purpose Language Understanding Systems. Advances in Neural Information Processing Systems (NeurIPS). arXiv:1905.00537.
4. Michael McCloskey & Neal J. Cohen (1989). Catastrophic Interference in Connectionist Networks: The Sequential Learning Problem. Psychology of Learning and Motivation.
5. James Kirkpatrick et al. (2017). Overcoming Catastrophic Forgetting in Neural Networks. Proceedings of the National Academy of Sciences (PNAS). arXiv:1612.00796. [Elastic Weight Consolidation (EWC)]
6. Neil Houlsby et al. (2019). Parameter-Efficient Transfer Learning for NLP. Proceedings of the 36th International Conference on Machine Learning (ICML). arXiv:1902.00751.
7. Xuezhi Wang et al. (2023). Self-Consistency Improves Chain of Thought Reasoning in Language Models. International Conference on Learning Representations (ICLR). arXiv:2203.11171.