neural_compressor.experimental.metric.coco_tools

Wrappers for third party pycocotools to be used within object_detection.

Note that nothing in this file is tensorflow related and thus cannot be called directly as a slim metric, for example.

TODO(jonathanhuang): wrap as a slim metric in metrics.py

Usage example: given a set of images with ids in the list image_ids and corresponding lists of numpy arrays encoding groundtruth (boxes and classes) and detections (boxes, scores and classes), where elements of each list correspond to detections/annotations of a single image, then evaluation (in multi-class mode) can be invoked as follows:

groundtruth_dict = coco_tools.ExportGroundtruthToCOCO(

image_ids, groundtruth_boxes_list, groundtruth_classes_list, max_num_classes, output_path=None)

detections_list = coco_tools.ExportDetectionsToCOCO(

image_ids, detection_boxes_list, detection_scores_list, detection_classes_list, output_path=None)

groundtruth = coco_tools.COCOWrapper(groundtruth_dict) detections = groundtruth.LoadAnnotations(detections_list) evaluator = coco_tools.COCOEvalWrapper(groundtruth, detections,

agnostic_mode=False)

metrics = evaluator.ComputeMetrics()

Module Contents

Classes

COCOWrapper

Wrapper for the pycocotools COCO class.

COCOEvalWrapper

Wrapper for the pycocotools COCOeval class.

Functions

ExportSingleImageGroundtruthToCoco(→ list)

Export groundtruth of a single image to COCO format.

ExportSingleImageDetectionBoxesToCoco(→ list)

Export detections of a single image to COCO format.

ExportSingleImageDetectionMasksToCoco(→ list)

Export detection masks of a single image to COCO format.
class neural_compressor.experimental.metric.coco_tools.COCOWrapper(dataset: Dict[str, Any], detection_type: str = 'bbox')[source]

Wrapper for the pycocotools COCO class.

dataset[source]

a dictionary holding bounding box annotations in the COCO format.

detection_type[source]

type of detections being wrapped. Can be one of [‘bbox’, ‘segmentation’]

class neural_compressor.experimental.metric.coco_tools.COCOEvalWrapper(groundtruth: pycocotools.coco.COCO = None, detections: pycocotools.coco.COCO = None, agnostic_mode=False, iou_type: str = 'bbox', iou_thrs: str | float = None, map_points=None)[source]

Wrapper for the pycocotools COCOeval class.

To evaluate, create two objects (groundtruth_dict and detections_list) using the conventions listed at http://mscoco.org/dataset/#format. Then call evaluation as follows:

groundtruth = coco_tools.COCOWrapper(groundtruth_dict) detections = groundtruth.LoadAnnotations(detections_list) evaluator = coco_tools.COCOEvalWrapper(groundtruth, detections,

agnostic_mode=False)

metrics = evaluator.ComputeMetrics()

neural_compressor.experimental.metric.coco_tools.ExportSingleImageGroundtruthToCoco(image_id: int | str, next_annotation_id: int, category_id_set: Set[str], groundtruth_boxes: numpy.array, groundtruth_classes: numpy.array, groundtruth_masks: numpy.array | None = None, groundtruth_is_crowd: numpy.array | None = None) → list[source]

Export groundtruth of a single image to COCO format.

This function converts groundtruth detection annotations represented as numpy arrays to dictionaries that can be ingested by the COCO evaluation API. Note that the image_ids provided here must match the ones given to ExportSingleImageDetectionsToCoco. We assume that boxes and classes are in correspondence - that is: groundtruth_boxes[i, :], and groundtruth_classes[i] are associated with the same groundtruth annotation.

In the exported result, “area” fields are always set to the area of the groundtruth bounding box.

Parameters:

  • image_id – a unique image identifier either of type integer or string.

  • next_annotation_id – integer specifying the first id to use for the groundtruth annotations. All annotations are assigned a continuous integer id starting from this value.

  • category_id_set – A set of valid class ids. Groundtruth with classes not in category_id_set are dropped.

  • groundtruth_boxes – numpy array (float32) with shape [num_gt_boxes, 4]

  • groundtruth_classes – numpy array (int) with shape [num_gt_boxes]

  • groundtruth_masks – optional uint8 numpy array of shape [num_detections, image_height, image_width] containing detection_masks.

  • groundtruth_is_crowd – optional numpy array (int) with shape [num_gt_boxes] indicating whether groundtruth boxes are crowd.

Returns:

A list of groundtruth annotations for a single image in the COCO format.

Raises:

ValueError – if (1) groundtruth_boxes and groundtruth_classes do not have the right lengths or (2) if each of the elements inside these lists do not have the correct shapes or (3) if image_ids are not integers

neural_compressor.experimental.metric.coco_tools.ExportSingleImageDetectionBoxesToCoco(image_id: int | str, category_id_set: Set[int], detection_boxes: numpy.array, detection_scores: numpy.array, detection_classes: numpy.array) → list[source]

Export detections of a single image to COCO format.

This function converts detections represented as numpy arrays to dictionaries that can be ingested by the COCO evaluation API. Note that the image_ids provided here must match the ones given to the ExporSingleImageDetectionBoxesToCoco. We assume that boxes, and classes are in correspondence - that is: boxes[i, :], and classes[i] are associated with the same groundtruth annotation.

Parameters:

  • image_id – unique image identifier either of type integer or string.

  • category_id_set – A set of valid class ids. Detections with classes not in category_id_set are dropped.

  • detection_boxes – float numpy array of shape [num_detections, 4] containing detection boxes.

  • detection_scores – float numpy array of shape [num_detections] containing scored for the detection boxes.

  • detection_classes – integer numpy array of shape [num_detections] containing the classes for detection boxes.

Returns:

A list of detection annotations for a single image in the COCO format.

Raises:

  • ValueError – if (1) detection_boxes, detection_scores and detection_classes

  • do not have the right lengths or (2) if each of the elements inside these –

  • lists do not have the correct shapes or (3) if image_ids are not integers. –

neural_compressor.experimental.metric.coco_tools.ExportSingleImageDetectionMasksToCoco(image_id: str | int, category_id_set: Set[int], detection_masks: numpy.array, detection_scores: numpy.array, detection_classes: numpy.array) → list[source]

Export detection masks of a single image to COCO format.

This function converts detections represented as numpy arrays to dictionaries that can be ingested by the COCO evaluation API. We assume that detection_masks, detection_scores, and detection_classes are in correspondence - that is: detection_masks[i, :], detection_classes[i] and detection_scores[i]

are associated with the same annotation.

Parameters:

  • image_id – unique image identifier either of type integer or string.

  • category_id_set – A set of valid class ids. Detections with classes not in

  • dropped. (category_id_set are) –

  • detection_masks – uint8 numpy array of shape [num_detections, image_height,

  • detection_masks. (image_width] containing) –

  • detection_scores – float numpy array of shape [num_detections] containing

  • masks. (the classes for detection) –

  • detection_classes – integer numpy array of shape [num_detections] containing

  • masks. –

Returns:

A list of detection mask annotations for a single image in the COCO format.

Raises:

  • ValueError – if (1) detection_masks, detection_scores and detection_classes

  • do not have the right lengths or (2) if each of the elements inside these –

  • lists do not have the correct shapes or (3) if image_ids are not integers. –