Diagnose optimization traces

Problem

Diagnose and give one action for each trace.

  1. Loss is 10,2,9,1,810,2,9,1,8; parameter norms grow and gradients stay large.
  2. Loss is stable; the gradient norm is 10−810^{-8}; the Hessian is positive definite.
  3. Data-fit loss is low, but the total objective is dominated by a very large L2L_2 penalty.
Reveal answer or reference solution

Trace 11 indicates an excessively large learning rate; reduce it or use a line search. Trace 22 is a converged strict local minimum; stop after checking tolerance. Trace 33 indicates overly strong regularization; reduce the penalty coefficient if validation performance supports the change. The data-fit term alone is not the stated total objective.

Outcome

Previous attempts

Loading attempts saved in this browser…

    Review Screening Results