Episode 12 · Planned
LoRA and QLoRA with held-out evaluation
Compare adapter training with fixed data, budgets and held-out quality evaluation.
Hold the dataset, prompt format, training budget and evaluation set fixed while comparing LoRA with QLoRA. In both cases the base weights remain frozen and low-rank adapters are trained; QLoRA quantizes the frozen base while gradients flow through it to the adapters. Record trainable parameters, base-weight precision, adapter configuration, peak memory, training time and checkpoint lineage. Evaluate both adapters on held-out task quality and serving behavior; successful training or lower loss alone does not establish improvement.
Evidence: deterministic lineage, learning curves, memory/time measurements, held-out quality results, uncertainty across repetitions where practical and inference cost for accepted adapters.