results package

Submodules

results.plots module

results.plots.plot_in_grid(cells: Sequence[matplotlib.figure.Figure | Image] | Sequence[Sequence[matplotlib.figure.Figure | Image]], border: bool = False, columns: Sequence[str] | None = None, figsize: Tuple[float, float] | None = None, rows: Sequence[str] | None = None, title: str | None = None, vertical: bool = False)

Create a matplotlib figure with the given cells arranged in a grid.

Parameters:

  • cells (Union[Sequence[Union["matplotlib.figure.Figure", PIL.Image.Image]], Sequence[Sequence[Union["matplotlib.figure.Figure", PIL.Image.Image]]], ]) – 1D list or 2D array of matplotlib figures or PIL images to display

  • border (bool, optional) – If True, draw borders around cell images, defaults to False

  • columns (Sequence[str], optional) – Labels for columns of the grid, defaults to None

  • figsize (Tuple[float, float], optional) – Size of the figure, defaults to None

  • rows (Sequence[str], optional) – Labels for rows of the grid, defaults to None

  • title (str, optional) – Title for the figure, defaults to None

  • vertical (bool, optional) – If True, arrange 1D cells vertically instead of horizontally, defaults to False

Returns:

Matplotlib figure

results.plots.plot_metrics(*runs: EvaluationResults, blacklist: Sequence[str] | None = None, metric_label: str = 'Metric', precision: int = 3, whitelist: Sequence[str] | None = None)

Create an HTML table of metrics from multiple runs.

Parameters:

  • *runs –

    EvaluationResults from runs to compare

  • blacklist (Sequence[str], optional) – Optional, list of metrics to exclude, defaults to None

  • metric_label (str, optional) – Optional, label for the metric column, defaults to “Metric”

  • precision (int, optional) – Optional, number of decimal places to display for floating point values, defaults to 3

  • whitelist (Sequence[str], optional) – Optional, list of metrics to include, defaults to None

Returns:

IPython HTML object for the table

results.plots.plot_params(*runs: EvaluationResults, blacklist: Sequence[str] | None = None, hide_same: bool = False, highlight_diff: bool = True, param_label: str = 'Parameter', whitelist: Sequence[str] | None = None)

Create an HTML table of parameters from multiple runs.

Parameters:

  • *runs –

    EvaluationResults from runs to compare

  • blacklist (Sequence[str], optional) – Optional, list of parameters to exclude, defaults to None

  • hide_same (bool, optional) – If True, hide parameters that are the same across all runs, defaults to False

  • highlight_diff (bool, optional) – If True, highlight parameters that are different across runs, defaults to True

  • param_label (str, optional) – Optional, label for the parameter column, defaults to “Parameter”

  • whitelist (Sequence[str], optional) – Optional, list of parameters to include, defaults to None

Returns:

IPython HTML object for the table

results.results module

Armory evaluation results

class results.results.BatchExports(batch_idx: int, artifacts: RunArtifacts)

Bases: object

Exported artifacts for a specific batch in an evaluation run

plot(filename: str | None = None, max_samples: int | None = None, samples: Sequence[int] | None = None, title: str | None = None) → matplotlib.figure.Figure

Create a matplotlib figure for image samples in the batch.

Parameters:

  • filename (str, optional) – Optional, image filename to plot. If not provided, the default image export for each sample will be plotted.

  • max_samples (int, optional) – Optional, maximum number of samples to plot. If not provided, all samples will be plotted.

  • samples (Sequence[int], optional) – Optional, specific sample indices to plot. If not provided, all samples will be plotted.

  • title (str, optional) – Optional, title for the plot

Returns:

Matplotlib figure

Return type:

matplotlib.figure.Figure

sample(sample_idx: int) → SampleExports

Retrieve exports for a specific sample index

property samples: Iterable[int]

Indices of exported samples in the batch

class results.results.ClassificationResults(sample: SampleExports)

Bases: object

Image classification results for a specific sample within a batch

plot(figure: matplotlib.pyplot.Figure | None = None, labels: Sequence[str] | None = None, top_k: int = 10) → matplotlib.figure.Figure

Create a matplotlib figure for the image classification input and predictions for this sample.

Parameters:

  • figure ("matplotlib.pyplot.Figure", optional) – Optional, existing matplotlib figure to use for plotting. If not provided, a new figure will be created.

  • labels (Sequence[str], optional) – Optional, class labels for the predictions. If not provided, class indices will be used.

  • top_k (int, optional) – Optional, number of top predictions to display, defaults to 10

Returns:

Matplotlib figure

Return type:

matplotlib.figure.Figure

class results.results.EvaluationResults(client: mlflow.client.MlflowClient, run: mlflow.entities.Run)

Bases: object

Armory evaluation results corresponding to a single MLFlow run

property artifacts: RunArtifacts

Run artifacts

batch(batch_idx: int) → BatchExports

Retrieve exports for a specific batch index

property batches: Iterable[int]

Indices of exported batches in the run

property children: Dict[str, EvaluationResults]

Child (nested) runs

property details: RunDataDict

Run details

classmethod for_last_run(experiment_id: str | None = None, experiment_name: str | None = None, max_search: int = 10) → EvaluationResults

Retrieve the evaluation results for the last run in a given MLFlow experiment (by name or by ID).

Parameters:

  • experiment_id (str, optional) – Optional, MLFlow experiment ID (if not using name), defaults to None

  • experiment_name (str, optional) – Optional, MLFlow experiment name (if not using ID), defaults to None

  • max_search (int, optional) – Optional, number of runs to search. This should only be necessary if the evaluations have more than 10 chains, defaults to 10

Returns:

EvaluationResults object

Return type:

EvaluationResults

classmethod for_run(run_id: str) → EvaluationResults

Retrieve the evaluation results for a given MLFlow run ID.

Parameters:

run_id (str) – MLFlow run ID

Returns:

EvaluationResults object

Return type:

EvaluationResults

property metrics: RunMetricsDict

Run model metrics

property params: RunDataDict

Run parameters

property run_id: str

MLFlow run ID

property run_name: str

MLFlow run name

property system_metrics: RunMetricsDict

Run system metrics

property tags: RunDataDict

Run tags

class results.results.MetricHistory(history: Sequence[mlflow.entities.Metric])

Bases: UserList

plot(figure: matplotlib.figure.Figure | None = None, timestamp: bool = False) → matplotlib.figure.Figure

Create a matplotlib figure for the metric history.

Parameters:

  • figure ("matplotlib.figure.Figure", optional) – Optional, existing matplotlib figure to use for plotting. If not provided, a new figure will be created.

  • timestamp (bool, optional) – Use timestamps for the x-axis instead of step numbers, defaults to False

Returns:

Matplotlib figure

Return type:

matplotlib.figure.Figure

class results.results.RunArtifact(client: mlflow.client.MlflowClient, run_id: str, artifact: mlflow.entities.FileInfo)

Bases: object

An individual artifact attached to an MLFlow run

property data: bytes

Raw data contents of the artifact file

property image: PIL.Image.Image

Artifact file as a PIL image

property json: Any

Artifact file as a parsed JSON object

property local_path: str

Local path to the downloaded artifact file. If the file is not already available locally, it will be downloaded from the MLFlow server.

class results.results.RunArtifacts(client: mlflow.client.MlflowClient, run_id: str, path: str = '', children: Dict[str, RunArtifact] | None = None)

Bases: object

Attached artifacts for a specific folder path in an MLFlow run

paths() → Iterable[str]

Paths of all artifacts under the current path

class results.results.RunDataDict(data: Dict[str, Any], title: str, key_label: str = 'key', value_label: str = 'value')

Bases: UserDict

Dictionary of run data that can be printed as a table

plot(format: ~typing.Callable[[~typing.Any], str] = <class 'str'>) → IPython.core.display.HTML

Create an HTML table for the dictionary.

Parameters:

format (Callable[[Any], str], optional) – Optional, function to format values for printing. Defaults to the built-in str function.

Returns:

IPython HTML object for the table

Return type:

IPython.core.display.HTML

table(console: rich.console.Console | None = None, format: ~typing.Callable[[~typing.Any], str] = <class 'str'>, title: str | None = None, **kwargs) → None

Print the contents of the dictionary in a rich table.

Parameters:

  • console ("rich.console.Console", optional) – Optional, rich console to use for printing. Defaults to the standard rich console.

  • format (Callable[[Any], str], optional) – Optional, function to format values for printing. Defaults to the built-in str function.

  • title (str, optional) – Optional, title for the table. Defaults to the title provided when the dictionary was initialized.

  • **kwargs –

    Keyword arguments forwarded to the rich table constructor

class results.results.RunMetricsDict(data: Dict[str, Any], title: str, client: mlflow.client.MlflowClient, run_id: str, key_label: str = 'metric', value_label: str = 'value')

Bases: RunDataDict

Dictionary of run metrics that can be printed as a table

history(key: str) → MetricHistory

Retrieve the history of a specific metric.

Parameters:

key (str) – Metric key

Returns:

Metric history object

Return type:

MetricHistory

plot(format: Callable[[Any], str] | None = None, precision: int = 3) → IPython.core.display.HTML

Create an HTML table for the metrics.

Parameters:

  • format (Callable[[Any], str], optional) – Optional, function to format values for printing. Defaults to a fixed-precision floating point formatter.

  • precision (int, optional) – Optional, number of decimal places to display for floating point values, defaults to 3

Returns:

IPython HTML object for the table

Return type:

IPython.core.display.HTML

table(console: rich.console.Console | None = None, format: Callable[[Any], str] | None = None, precision: int = 3, title: str | None = None, **kwargs) → None

Print the contents of the dictionary in a rich table.

Parameters:

  • console ("rich.console.Console", optional) – Optional, rich console to use for printing. Defaults to the standard rich console.

  • format (Callable[[Any], str], optional) – Optional, function to format values for printing. Defaults to a fixed-precision floating point formatter.

  • precision (int, optional) – Optional, number of decimal places to display for floating point values, defaults to 3

  • title (str, optional) – Optional, title for the table. Defaults to the title provided when the dictionary was initialized, defaults to None

  • **kwargs –

    Keyword arguments forwarded to the rich table constructor

class results.results.SampleExports(batch_idx: int, sample_idx: int, artifacts: RunArtifacts)

Bases: object

Exported artifacts for a specific sample within a batch

property classification: ClassificationResults

Sample as an image classification sample

property exports: Iterable[str]

Artifact filenames for the sample

property imagename: str

Default image export for the sample

property metadata: SampleMetadata

Metadata properties for the sample

class results.results.SampleMetadata(batch_idx: int, sample_idx: int, artifact: RunArtifact)

Bases: object

Metadata properties for a specific sample within a batch

property json: Any

Sample metadata as a parsed JSON object

results.utils module

results.utils.get_mlflow_client() → mlflow.client.MlflowClient

Create an MLFlow client

Module contents