What Changed When I Increased LoRA Rank from 8 to 32

LoRA rank is easy to treat as a quality dial: increase it when the adapter misses details, reduce it when the file is too large. That shortcut hides the useful question. Rank changes the capacity of the update matrices, but the outcome still depends on the dataset, target modules, learning rate, and evaluation set.

I compared rank 8, 16, and 32 on the same instruction-tuning task. The base model, seed, prompt template, dataset split, target modules, batch size, and number of optimizer steps remained fixed. The evaluation set was frozen before the first run. That constraint matters. If prompts or labels move between runs, a rank experiment becomes a comparison of several variables at once.

Keep the comparison narrow

Each record asked the model to return a small JSON object with a category, a priority, and a concise summary. The training set included common support categories plus a smaller set of escalation cases. I used the same preprocessing output for every run and recorded its digest with each configuration.

The three configurations were deliberately close:

Alpha scaled with rank so the effective multiplier stayed comparable. That does not make the runs identical, but it prevents an accidental comparison between a small rank change and a much larger scaling change.

The adapter parameter count grows roughly with rank and the dimensions of the modules being adapted. The exact total depends on the model architecture and target list, but the relationship is linear for a fixed set of linear layers.

from dataclasses import dataclass

@dataclass(frozen=True)
class LinearLayer:
    input_features: int
    output_features: int

def lora_parameters(layer: LinearLayer, rank: int) -> int:
    return rank * (layer.input_features + layer.output_features)

layers = [
    LinearLayer(4096, 4096),
    LinearLayer(4096, 4096),
    LinearLayer(4096, 11008),
    LinearLayer(11008, 4096),
]

for rank in (8, 16, 32):
    total = sum(lora_parameters(layer, rank) for layer in layers)
    print(f"rank={rank:>2} parameters={total:,}")

This script is not a complete Qwen parameter report. It is a transparent way to see why doubling rank doubles the update capacity when the adapted layers remain unchanged.

Train with one variable changed

The paired commands below differ only in rank and alpha. They use the same dataset, validation file, base model, seed, and run duration.

dock finetune \
  --base Qwen/Qwen2.5-7B-Instruct \
  --data ./support-train.jsonl \
  --validation-data ./support-validation.jsonl \
  --output qwen7b-ticket-triage-r8 \
  --rank 8 \
  --alpha 16 \
  --dropout 0.05 \
  --learning-rate 2e-4 \
  --max-seq-length 1024 \
  --epochs 3 \
  --seed 42

dock finetune \
  --base Qwen/Qwen2.5-7B-Instruct \
  --data ./support-train.jsonl \
  --validation-data ./support-validation.jsonl \
  --output qwen7b-ticket-triage-r32 \
  --rank 32 \
  --alpha 64 \
  --dropout 0.05 \
  --learning-rate 2e-4 \
  --max-seq-length 1024 \
  --epochs 3 \
  --seed 42

A rank experiment should also keep decoding fixed. A change to temperature, top-p, stop tokens, or JSON repair logic can make one checkpoint look better without changing the learned adapter at all.

Inspect more than loss

Training loss is useful for detecting divergence or an obviously underfit run. It is not enough to choose a release. I checked four areas for each checkpoint:

• Held-out task accuracy, including separate slices for rare queues.
• Schema validity before any response repair.
• Formatting stability across prompts with extra whitespace, quoted text, and missing fields.
• Memorisation signals, such as unusually specific summaries on near-duplicate examples.

Rank 8 was often adequate when the labels were clean and the desired output was short. Its limitation appeared on examples that combined a routing decision with a conditional escalation note. Rank 16 provided more room for that interaction without immediately expanding the release surface. Rank 32 was worth trying when the validation set showed a consistent pattern of missing distinctions rather than random errors.

A useful review table records outcomes without pretending that one score settles the decision:

The table intentionally contains no universal winner. A larger adapter earns its size only when its advantage appears on frozen validation examples and survives a manual error review.

Reject capacity that does not change the decision

The rank-32 checkpoint should be rejected when it produces a larger artifact but does not improve the cases that matter. That includes runs where validation loss falls slightly while escalation recall, JSON validity, and difficult-category accuracy remain unchanged. It also includes runs that improve a few examples by becoming more verbose or by copying unusual wording from training records.

One additional check is output variance. Run the same held-out prompts with the same deterministic decoding settings and compare the result after harmless input changes: extra spaces, a reordered non-semantic field, or a blank optional field. A useful adapter should preserve the routing decision when the meaning is unchanged. If rank 32 becomes more sensitive to these presentation details, the apparent capacity gain may be a formatting dependency rather than a task improvement.

The release record should also state which modules were adapted. Increasing rank while silently expanding the target-module list makes the experiment impossible to interpret. Keep those choices separate: first test rank with a fixed target list, then test a different target list only if the remaining errors point to a capability gap that rank did not address.

This discipline also makes rollback straightforward. Keep the smaller checkpoint and its evaluation record available until the larger version has passed the same release gate on new data. Capacity is a cost that should be justified by a repeatable decision, not by a more impressive configuration label.

When rank 32 performs better only on examples close to the training corpus, the next step is not another rank increase. Audit duplicates, label ambiguity, and split boundaries first. Capacity cannot repair a taxonomy that asks the model to infer inconsistent policy.

For a new task, I start with rank 16 when the output combines a decision and structured text. I move down to rank 8 when the task is narrow and the evaluation set remains stable. I test rank 32 only with a written hypothesis about which failure mode more capacity should address. That sequence keeps the experiment interpretable and makes the registry version meaningful when a release is promoted.