Parameters and output gradient

Tutor me on https://mlprep.iwase.dev/item/ml-screen-neural-networks-advanced-softmax/. This is a screening item. If window.mlPrepAgent is available, read attempts for item ml-screen-neural-networks-advanced-softmax before tutoring. Inspect my attempt, keep the item ID, and do not reveal the full answer first. After a real attempt, append its record with recorded_by agent, include agent_session_id when available, and read it back.

Problem

A dense classifier has $4$ inputs, $3$ hidden units with biases, and $2$ softmax outputs with biases. Count the trainable scalars. For $h=(1,2,0)$ , $p=(0.25,0.75)$ , and one-hot $y=(1,0)$ , compute the cross-entropy gradients for the output logits, output bias, and output weight matrix.

Reveal answer or reference solution

The first layer has $3\cdot4+3=15$ parameters, and the output layer has $2\cdot3+2=8$ , for a total of $23$ . Softmax cross-entropy gives

\frac{\partial L}{\partial z}=p-y=(-0.75,0.75),

which is also the bias gradient. The weight gradient is

\frac{\partial L}{\partial W}=(p-y)h^{\mathsf T}=\begin{bmatrix}-0.75&-1.5&0\\0.75&1.5&0\end{bmatrix}.

Previous attempts

Loading attempts saved in this browser…

Review Screening Results

Parameters and output gradient

Problem

Previous attempts

Appears in