What values are returned from model.evaluate() in Keras?
Asked Answered
H

2

53

I've got multiple outputs from my model from multiple Dense layers. My model has 'accuracy' as the only metric in compilation. I'd like to know the loss and accuracy for each output. This is some part of my code.

scores = model.evaluate(X_test, [y_test_one, y_test_two], verbose=1)

When I printed out the scores, this is the result.

[0.7185557290413819, 0.3189622712272771, 0.39959345855771927, 0.8470299135229717, 0.8016634374641469]

What are these numbers represent?

I'm new to Keras and this might be a trivial question. However, I have read the docs from Keras but I'm still not sure.

Halmahera answered 12/7, 2018 at 7:36 Comment(0)
B
62

Quoted from evaluate() method documentation:

Returns

Scalar test loss (if the model has a single output and no metrics) or list of scalars (if the model has multiple outputs and/or metrics). The attribute model.metrics_names will give you the display labels for the scalar outputs.

Therefore, you can use metrics_names property of your model to find out what each of those values corresponds to. For example:

from keras import layers
from keras import models
import numpy as np

input_data = layers.Input(shape=(100,)) 
out_1 = layers.Dense(1)(input_data)
out_2 = layers.Dense(1)(input_data)

model = models.Model(input_data, [out_1, out_2])
model.compile(loss='mse', optimizer='adam', metrics=['mae'])

print(model.metrics_names)

outputs the following:

['loss', 'dense_1_loss', 'dense_2_loss', 'dense_1_mean_absolute_error', 'dense_2_mean_absolute_error']

which indicates what each of those numbers you see in the output of evaluate method corresponds to.

Further, if you have many layers then those dense_1 and dense_2 names might be a bit ambiguous. To resolve this ambiguity, you can assign names to your layers using name argument of layers (not necessarily on all of them but only on the input and output layers):

# ...
out_1 = layers.Dense(1, name='output_1')(input_data)
out_2 = layers.Dense(1, name='output_2')(input_data)
# ...

print(model.metrics_names)

which outputs a more clear description:

['loss', 'output_1_loss', 'output_2_loss', 'output_1_mean_absolute_error', 'output_2_mean_absolute_error']
Bittencourt answered 12/7, 2018 at 10:31 Comment(2)
I have retuned recall, precision, AUC and accuracy which is fine, but how can we access each metric separately so that we can use them wherever we want? I have tried subscripting and dot syntax to access the properties or indices.Muttra
It's also possible to add return_dict=True to evaluate method and you will get the name of the metrics!Aarhus
F
1

We should be clear that the "loss" figure is the sum of ALL the losses calculated for each item in the x_test array. x_test would contain your test data and y_test would contain your labels. The loss figure is the sum of ALL the losses, not just one loss from one item in the x_test array.

Frigidaire answered 25/11, 2021 at 14:53 Comment(0)

© 2022 - 2024 — McMap. All rights reserved.