what does cfg.MODEL.ROI_HEADS.SCORE_THRESH_TEST = 0.5 do in detectron2?

Question

Hope you're doing great!

I didn't really understand these 2 lines from the detectron2 colab notebook tutorial, I tried looking in the official documentation but i didn't understand much, can someone please explain this to me :

cfg.MODEL.ROI_HEADS.SCORE_THRESH_TEST = 0.5  # set threshold for this model
# Find a model from detectron2's model zoo. You can use the https://dl.fbaipublicfiles... url as well
cfg.MODEL.WEIGHTS = model_zoo.get_checkpoint_url("COCO-InstanceSegmentation/mask_rcnn_R_50_FPN_3x.yaml")

I thank you in advance and wish you a great day!

Dewey

Answer 1

The cfg.MODEL.ROI_HEADS.SCORE_THRESH_TEST value is the threshold used to filter out low-scored bounding boxes predicted by the Fast R-CNN component of the model during inference/test time.

Basically, any prediction with a confidence score above the threshold value is kept, and the remaining are discarded.

This thresholding can be seen in the Detectron2 code here.

def fast_rcnn_inference_single_image(
    boxes,
    scores,
    image_shape: Tuple[int, int],
    score_thresh: float,
    nms_thresh: float,
    topk_per_image: int,
):

    ### clipped code ###

    # 1. Filter results based on detection scores. It can make NMS more efficient
    #    by filtering out low-confidence detections.
    filter_mask = scores > score_thresh  # R x K

    ### clipped code ###

You can also see here to confirm that that parameter value originates from the config.

class FastRCNNOutputLayers(nn.Module):
    """
    Two linear layers for predicting Fast R-CNN outputs:
    1. proposal-to-detection box regression deltas
    2. classification scores
    """
    
    ### clipped code ###

    @classmethod
    def from_config(cls, cfg, input_shape):
        return {
            "input_shape": input_shape,
            "box2box_transform": Box2BoxTransform(weights=cfg.MODEL.ROI_BOX_HEAD.BBOX_REG_WEIGHTS),
            # fmt: off
            "num_classes"           : cfg.MODEL.ROI_HEADS.NUM_CLASSES,
            "cls_agnostic_bbox_reg" : cfg.MODEL.ROI_BOX_HEAD.CLS_AGNOSTIC_BBOX_REG,
            "smooth_l1_beta"        : cfg.MODEL.ROI_BOX_HEAD.SMOOTH_L1_BETA,
            "test_score_thresh"     : cfg.MODEL.ROI_HEADS.SCORE_THRESH_TEST,
            "test_nms_thresh"       : cfg.MODEL.ROI_HEADS.NMS_THRESH_TEST,
            "test_topk_per_image"   : cfg.TEST.DETECTIONS_PER_IMAGE,
            "box_reg_loss_type"     : cfg.MODEL.ROI_BOX_HEAD.BBOX_REG_LOSS_TYPE,
            "loss_weight"           : {"loss_box_reg": cfg.MODEL.ROI_BOX_HEAD.BBOX_REG_LOSS_WEIGHT},
            # fmt: on
        }

    ### clipped code ###

Champion

Well explained, i got it, thank you so much !! – Dewey

thanks a lot for the answer! do you know if there is any documentation on those variables? – Teenager

@Teenager There is a reference for all the config parameters here if that's what you're looking for? – Champion

Is there an equivalent setting but for the training session? Like discard or punish results below the equivalent of cfg.MODEL.ROI_HEADS.SCORE_THRESH_TEST? – Wetzell

@Wetzell I think what you are asking for is equivalent to deleting the objects you don't want from your training set. I don't think it is possible to punish objects during training. – Dewey