MSRS Explained: Shared and Private Subspaces for Multi-Attribute AI Steering

How can one internal edit improve several model behaviors without blindly adding steering vectors? We teach MSRS's learned low-rank coordinate replacement, SVD-derived shared and private bases, projection subtraction, state-dependent gates, and three-part training objective. Frozen backbone weights are distinct from trainable R, W, bias and mask-network parameters. Exact citation: Xinyan Jiang, Lin Zhang, Jiayi Zhang, Qingsong Yang, Guimin Hu, Di Wang, and Lijie Hu. Adaptive Multi-Subspace Representation Steering for Attribute Alignment in Large Language Models. arXiv:2508.10599v4. PDF version stamp 25 April 2026; preprint footer says 28 April 2026. https://arxiv.org/abs/2508.10599v4 Local PDF: 21 pages, all reviewed including references and Appendices A-J; source.txt consulted and all pages independently extracted/read. Central mathematics and headline tables visually checked; not every plot digitized. SHA-256: 388217f740f590c3505275490d48ca639687b14a531d8463fafe64f844708be9. No separate public-PDF byte comparison claimed. Checked TOYS, not model runs: rank-two R selects first two axes of h=(2,3,4); target coordinates (5,1), delta (3,-2), gates (.5,1), result (3.5,1,4). A second toy removes shared and prior-private projections to recover three perpendicular axes, then composes shared/private changes. The full-rank composition toy checks geometry, not compression. Shared rank uses the paper's SUM of singular values at 60%, not squared singular-value energy. Authors' Table 1 Llama3-8B-Instruct: MC2 45.63 to 56.32; BBQ accuracy .608 to .645. ITI still wins MC1, 36.50 versus MSRS34.91. These are trade-offs, not every-metric wins. Baselines include ICL, CAA, ITI, ReFT, CS-ReFT and MAT-STEER. Three seeds, Adam .0009, batch4, regularizers .3/.5 and last-five-token main setup. Layer/hyperparameter selection uses validation splits; precise dataset splits and matched baseline tuning budgets are incompletely reported. This is not proof of arbitrary unseen-attribute generalization. Public limits: orthogonality does NOT guarantee semantic independence, causal isolation or safety. Learned-R constraint enforcement, private-rank cutoff, rank-budget reconciliation and block-to-coordinate gate expansion are not fully specified. Frobenius matrix alignment is basis-sensitive and does not itself enforce exact orthogonality. The printed task-loss formula abbreviates the downstream-logit path; we label that dimensional clarification. The detailed extraction uses activation means, despite introduction wording about differences. Reporting caveats: Llama3 GLUE is .775 in Table2 but .7578 in Table7. Appendix gating/position variants also differ from headline cells; figures are not silently reconciled. Mistral MMLU falls .618 to .613. Important-token intervention does not improve every metric. Verbosity prose and table arrows differ in interpretation. Many-attribute gains weaken, with baseline crossings around nine/ten; five sampled combinations per target/count are not all possible combinations. Contamination tipping point near .61 is one helpfulness/coherence mixing study, not a universal threshold; fits only cover up to ten attributes. Table4 latency rises102.44 to120.35ms/token, about17.5%; offline extraction/training cost extra, and request-level appendix timings are a different setup. Safety discussion is high-level only; no actionable attack prompts or implementation. Independent explanation; no author endorsement. Authors' results, no experiments reproduced. AI-generated narration configured for local Kokoro af_bella, en-us, speed1.0.