metric module

class metric.Metric(metric: TorchMetric, spec: DataSpecification | None = None, record_as_artifact: bool = True, record_as_metrics: Iterable[str] | None = None)

Bases: Trackable, Module, ABC

Base class for an Armory-compatible metric.

clone() → Self

Creates a clone of the metric.

Returns:

Clone of the metric

Return type:

Self

compute()

Computes the metric value(s).

get_scalars(value: Any) → Dict[str, float]

reset()

Resets the metric.

classmethod to_json(value: Any) → Any

training: bool

abstract update(batch: Batch) → None

Updates the metric with a batch from the evaluation.

Parameters:

batch (Batch) – Batch from the evaluation

class metric.PerturbationMetric(metric: TorchMetric, spec: DataSpecification | None = None, record_as_artifact: bool = True, record_as_metrics: Iterable[str] | None = None)

Bases: Metric

A metric based on comparing the unperturbed input data against the final perturbed data used as input to the model.

The wrapped metric’s update method must accept two values:

• The unperturbed input data

• The final perturbed input data

Example:

from armory.metric import PerturbationMetric
from armory.metrics.perturbation import PerturbationNormMetric

metric = PerturbationMetric(PerturbationNormMetric())

training: bool

update(batch: Batch) → None

Updates the metric with a batch from the evaluation.

Parameters:

batch (Batch) – Batch from the evaluation

class metric.PredictionMetric(metric: TorchMetric, spec: DataSpecification | None = None, record_as_artifact: bool = True, record_as_metrics: Iterable[str] | None = None)

Bases: Metric

A metric based on comparing the natural, or ground truth, targets against the model’s predictions.

The wrapped metric’s update method must accept two values:

• The model’s predictions

• The ground truth targets

Example:

from torchmetrics.classification import Accuracy
from armory.metric import PredictionMetric

metric = PredictionMetric(Accuracy())

training: bool

update(batch: Batch) → None

Updates the metric with a batch from the evaluation.

Parameters:

batch (Batch) – Batch from the evaluation