How many trainable weights does your model have?

You are going to train a DNN regression model with Keras APIs using this code:

How many trainable weights does your model have? (The arithmetic below is correct.)
A . 501*256+257*128+2 161154
B . 500*256+256*128+128*2 161024
C . 501*256+257*128+128*2161408
D . 500*256*0 25+256*128*0 25+128*2 40448

Answer: B

Explanation:

The number of trainable weights in a DNN regression model with Keras APIs can be calculated by multiplying the number of input units by the number of output units for each layer, and adding the number of bias units for each layer. The bias units are usually equal to the number of output units, except for the last layer, which does not have bias units if the activation function is softmax1. In this code, the model has three layers: a dense layer with 256 units and relu activation, a dropout layer with 0.25 rate, and a dense layer with 2 units and softmax activation. The input shape is 500.

Therefore, the number of trainable weights is:

For the first layer: 500 input units * 256 output units + 256 bias units 128256

For the second layer: The dropout layer does not have any trainable weights, as it only randomly sets some of the input units to zero to prevent overfitting2.

For the third layer: 256 input units * 2 output units + 0 bias units 512

The total number of trainable weights is 128256 + 512 161024. Therefore, the correct answer is B.

Reference: How to calculate the number of parameters for a Convolutional Neural Network?

Dropout (keras.io)