Evaluation Utils

used to calculate the detector performance currently support mAP for object detection

class eyewitness.evaluation.BboxMAPEvaluator(iou_threshold=0.5, dataset_mode='TEST_ONLY', logging_frequency=100)

Bases: eyewitness.evaluation.Evaluator

evaluate the bbox mAP score

static calculate_average_precision(recall, precision)

calculate_label_ap(valid_labels, detected_objs, gt_objs, gt_label_count)

refactor the evaluation from https://github.com/rafaelpadilla/Object-Detection-Metrics

evaluation_method

class eyewitness.evaluation.Evaluator

Bases: object

evaluate(detector, dataset)

evaluation_method

A BboxMAPEvaluator Example

# a evaluation example with yolov3 detector
# https://github.com/penolove/keras-yolo3/blob/eyeWitnessWrapper/eyewitness_evaluation.py
from eyewitness.config import DATASET_TEST_ONLY
dataset_folder = 'VOC2007'
dataset_VOC_2007 = BboxDataSet(dataset_folder, 'VOC2007')
object_detector = YoloV3DetectorWrapper(args, threshold=0.0)
bbox_map_evaluator = BboxMAPEvaluator(dataset_mode=DATASET_TEST_ONLY)
# which will lead to ~0.73
print(bbox_map_evaluator.evaluate(object_detector, dataset_VOC_2007)['mAP'])