Methods that perform very well under one challenge (e.g., background motion) may not perform well in the presence of another challenge (e.g., strong shadows or night videos). In order to gauge performance and rank methods, we rely on the following widely used metrics:

  • AGE: (Average Gray-level Error). Average of the gray-level absolute difference between GT and the computed background (CB) image.
  • pEPs: (Percentage of Error Pixels). Percentage of EPs (number of pixels in CB whose value differs from the value of the corresponding pixel in GT by more than a threshold) with respect to the total number of pixels in the image.
  • pCEPS: (Percentage of Clustered Error Pixels). Percentage of CEPs (number of pixels whose 4-connected neighbors are also error pixels) with respect to the total number of pixels in the image.
  • MSSSIM: (MultiScale Structural Similarity Index). Estimate of the perceived visual distortion.
  • PSNR: (Peak-Signal-to-Noise-Ratio) Amounts to 10log_10((L-1)^2/MSE) where L is the maximum number of grey levels and MSE is the Mean Squared Error between GT and CB images.
  • CQM: (Color image Quality Measure). Based on a reversible transformation of the YUV color space and on the PSNR computed in the single YUV bands. It assumes values in db and the higher the CQM value, the better is the background estimate.

These metrics are reported for SBMnet dataset and allow to identify algorithms that are robust across various challenges. The source code to compute all performance metrics is provided in UTILITIES.