fig, (ax1, ax2) = plt.subplots(1, 2, sharey=True, sharex=True, figsize=(16, 5))
nn_plot(hist[17], 200, subplot=ax1, loss=True, title="Reference: lambda = 0.01")
nn_plot(hist[23], 200, subplot=ax2, loss=True, title="lambda = 0.1")

fig, (ax1, ax2) = plt.subplots(1, 2, sharey=True, sharex=True, figsize=(16, 5))
nn_plot(hist[17], 200, subplot=ax1, acc=True, title="Reference: lambda = 0.01")
nn_plot(hist[23], 200, subplot=ax2, acc=True, title="lambda = 0.1")

We can see through the graphics that by increasing $\lambda$ the model got worse. The default value 0.01 usually is a good value for the general cases.

Full training set: L2 Regularization: $\lambda = 0.005$

We will proceed by analyzing the full training set with a $\lambda = 0.005$, which would be half the default value.

model.append(tf.keras.models.Sequential([
    tf.keras.layers.Dense(100, kernel_regularizer=tf.keras.regularizers.l2(0.005),
                          activation='relu', input_shape=(784,), name='hidden_1_layer'),
    tf.keras.layers.Dense(100, kernel_regularizer=tf.keras.regularizers.l2(0.005),
                          activation='relu', name='hidden_2_layer'),
    tf.keras.layers.Dense(10, activation='softmax', name='output_layer')
]))

nn_compile(model[-1], params[3])