Profile-aware diagnostics of discarded parameter-Hessian mass
Maxim Bolshim, Alexander Kugaevskikh
Abstract
A discarded parameter-Hessian fraction depends on the diagonal profile and parameter partition; it is not the Fisher information discarded by K-FAC. We compare off-block squared Frobenius mass with group-size and measured diagonal-profile references at specified weights, inputs and coordinates. Similar initial fractions in ResNet-18 and its skip-free twin accompany different profiles. On the same saved trained ResNet-18 states, changing from stage groups to layers moves estimated mass from below to above the profile envelope. An approximate joint conditional Monte Carlo screen resolves this crossing in 6/6 seed/input cells; the skip-free network has 2 resolved and 4 unresolved comparisons. A separate comparison of nine saved states under two geometry draws splits the full model residual into off-group Hessian action and its remainder: gradient, within-group action and damping. At unit correction scale, off-group input alone increases loss in 17 of 18 state-geometry comparisons; in nine of those cases, the full candidate has lower loss than the remainder-only candidate. After correction-norm matching, seven of the ten comparisons that initially favoured full favour the remainder. These finite-loss contrasts depend on scale and do not establish a restoration benefit.