fig, (ax1, ax2) = plt.subplots(1, 2, sharey=True, sharex=True, figsize=(16, 5))
nn_plot(hist[16], 200, subplot=ax1, loss=True, title="Reference: hidden layer: 30 nodes")
nn_plot(hist[17], 200, subplot=ax2, loss=True, title="hidden layer: 100 nodes")

fig, (ax1, ax2) = plt.subplots(1, 2, sharey=True, sharex=True, figsize=(16, 5))
nn_plot(hist[16], 200, subplot=ax1, acc=True, title="Reference: hidden layer: 30 nodes")
nn_plot(hist[17], 200, subplot=ax2, acc=True, title="hidden layer: 100 nodes")

Both models are very similar, however the one with 100 nodes in the hidden layer should have slightly better results. It also takes more time to execute and more epochs to converge, since it has approximately three times more parameters to learn. By increasing the number of nodes means that the model can learn more peculiarities from the predictors, however the number can't be exaggerated big, or the model will become overfitted.

2 hidden layers: 30 nodes each

Our next topology to analyze will be two hidden layers, instead of one. We will begin with 30 nodes on each hidden layer.

model.append(tf.keras.models.Sequential([
    tf.keras.layers.Dense(30, kernel_regularizer=tf.keras.regularizers.l2(0.01),
                          activation='relu', input_shape=(784,), name='hidden_1_layer'),
    tf.keras.layers.Dense(30, kernel_regularizer=tf.keras.regularizers.l2(0.01),
                          activation='relu', name='hidden_2_layer'),
    tf.keras.layers.Dense(10, activation='softmax', name='output_layer')
]))

nn_compile(model[-1], params[3])

model[-1].summary()

Model: "sequential_19"
_________________________________________________________________
 Layer (type)                Output Shape              Param #   
=================================================================
 hidden_1_layer (Dense)      (None, 30)                23550     
                                                                 
 hidden_2_layer (Dense)      (None, 30)                930       
                                                                 
 output_layer (Dense)        (None, 10)                310       
                                                                 
=================================================================
Total params: 24,790
Trainable params: 24,790
Non-trainable params: 0
_________________________________________________________________

tf.keras.utils.plot_model(model[-1], show_shapes=True, show_layer_names=True)