Results for SBMnet 2016

Results, all categories combined.

Click on method name for more details.

Method Average ranking Average ranking across categories Average     AGE  Average     pEPs  Average   pCEPS  Average   MSSSIM  Average   PSNR     CQM      
Temporal median filter [2] 15.67 11.13 8.2761 0.0984 0.0546 0.9130 27.5364 28.4434
Photomontage [3] 10.17 14.13 7.1950 0.0686 0.0257 0.9189 28.0113 28.8719
RSL2011 [4] 20.83 21.63 9.0443 0.1008 0.0497 0.8891 25.8051 26.7986
FC-FlowNet [5] 18.83 16.50 9.1131 0.1128 0.0599 0.9162 26.9559 27.8767
LaBGen [6] 7.00 10.38 6.7090 0.0631 0.0265 0.9266 28.6396 29.4668
LaBGen-P [7] 8.67 11.00 7.0738 0.0706 0.0319 0.9278 28.4660 29.3196
RMR [8] 22.67 20.88 9.5363 0.1176 0.0582 0.8790 26.5217 27.4549
SC-SOBS-C4 [9] 11.50 13.13 7.5183 0.0711 0.0242 0.9160 27.6533 28.5601
MAGRPCA [10] 12.67 12.63 8.3132 0.0994 0.0567 0.9401 28.4556 29.3152
TMFG [11] 18.67 12.50 9.4020 0.1051 0.0566 0.9043 27.1347 28.0530
Bidirectional Analysis and Consensus Voting [12] 16.83 16.00 8.5816 0.0724 0.0257 0.9078 26.1018 27.1000
Bidirectional Analysis [13] 15.17 14.13 8.3449 0.0756 0.0181 0.9085 26.1722 27.1637
BE-AAPSA [14] 15.50 15.00 7.9086 0.0873 0.0447 0.9127 27.0714 27.9811
MSCL [15] 1.67 5.75 5.9547 0.0524 0.0171 0.9410 30.8952 31.7049
DECOLOR [16] 26.17 23.25 11.6340 0.1667 0.1098 0.9011 25.5717 26.4989
3-Term Decomposition [17] 26.00 23.38 11.5438 0.1614 0.0993 0.8863 25.3154 26.2844
BRTF [18] 28.17 27.50 11.9229 0.1614 0.0991 0.8799 24.9256 25.9302
RFSA [19] 30.83 28.00 13.2359 0.1717 0.1092 0.8723 24.6671 25.6682
RMAMR [20] 19.67 18.75 9.6995 0.1243 0.0770 0.9258 26.5380 27.4680
SSGoDec [21] 26.50 24.13 11.5934 0.1584 0.0974 0.8854 24.9954 25.9955
GRASTA [22] 28.67 24.88 11.9362 0.1696 0.1085 0.8833 24.9914 25.9995
GOSUS [23] 28.00 24.63 12.1183 0.1621 0.1002 0.8817 25.0354 26.0370
BEWiS [24] 6.17 8.25 6.7094 0.0592 0.0266 0.9282 28.7728 29.6342
LaBGen-OF [25] 3.00 5.88 6.1897 0.0566 0.0232 0.9412 29.8957 30.7006
NExBI [26] 9.00 14.38 6.7778 0.0671 0.0227 0.9196 27.9944 28.8810
ABM [27] 17.83 16.63 8.4182 0.0865 0.0387 0.9012 26.3536 27.3215
LaBGen-P-Semantic (CV+U) [28] 9.67 9.63 7.3890 0.0761 0.0357 0.9267 28.5050 29.3829
LaBGen-P-Semantic (MP+U) [29] 12.17 10.75 7.9731 0.0820 0.0394 0.9212 28.3234 29.1992
SPMD [30] 1.83 6.38 6.0985 0.0487 0.0154 0.9412 29.8439 30.6499
FSBE [31] 4.33 10.50 6.6204 0.0605 0.0217 0.9373 29.3378 30.1777
AAPSA [1] 22.17 21.63 9.2044 0.1057 0.0523 0.9000 25.3947 26.3021

Results, for the basic category.

Click on method name for more details.

Method Average ranking Average     AGE  Average     pEPs  Average   pCEPS  Average   MSSSIM  Average   PSNR     CQM      
Temporal median filter [2] 5.33 3.8269 0.0139 0.0034 0.9804 33.7085 34.3342
Photomontage [3] 13.17 4.4856 0.0226 0.0039 0.9719 32.3208 32.9621
RSL2011 [4] 16.67 4.5546 0.0269 0.0081 0.9660 31.6767 32.4654
FC-FlowNet [5] 18.50 5.5856 0.0327 0.0142 0.9636 30.4121 31.2285
LaBGen [6] 9.17 3.9012 0.0154 0.0045 0.9742 32.9009 33.5733
LaBGen-P [7] 8.67 3.9712 0.0156 0.0041 0.9749 33.2445 33.8797
RMR [8] 23.17 5.8867 0.0480 0.0167 0.9400 29.6069 30.4044
SC-SOBS-C4 [9] 12.83 4.3598 0.0200 0.0033 0.9728 32.1766 32.8665
MAGRPCA [10] 21.33 8.9576 0.1124 0.0882 0.9669 29.6430 30.4461
TMFG [11] 4.00 3.8063 0.0131 0.0031 0.9803 33.7483 34.3541
Bidirectional Analysis and Consensus Voting [12] 11.83 4.1421 0.0159 0.0019 0.9723 31.9277 32.6950
Bidirectional Analysis [13] 10.00 4.1075 0.0161 0.0017 0.9736 32.3092 33.0339
BE-AAPSA [14] 20.83 5.6842 0.0472 0.0273 0.9626 30.1101 30.9398
MSCL [15] 1.33 3.4019 0.0112 0.0019 0.9807 35.1206 35.6507
DECOLOR [16] 27.50 9.7332 0.1203 0.0920 0.9523 27.9299 28.8626
3-Term Decomposition [17] 26.83 9.2635 0.1200 0.0927 0.9483 28.2310 29.0882
BRTF [18] 24.33 9.5385 0.1140 0.0876 0.9621 28.4655 29.3246
RFSA [19] 25.67 9.6654 0.1183 0.0915 0.9609 28.3326 29.1893
RMAMR [20] 21.33 8.9576 0.1124 0.0882 0.9669 29.6430 30.4461
SSGoDec [21] 29.67 12.6639 0.1706 0.1343 0.9249 26.6982 27.5852
GRASTA [22] 29.17 12.3337 0.1722 0.1372 0.9414 26.8165 27.7176
GOSUS [23] 31.00 15.8439 0.1753 0.1425 0.9075 26.4462 27.3629
BEWiS [24] 12.00 4.0673 0.0162 0.0058 0.9770 32.2327 33.0035
LaBGen-OF [25] 4.17 3.8421 0.0118 0.0033 0.9796 33.7714 34.3918
NExBI [26] 17.17 4.7466 0.0234 0.0058 0.9630 31.4990 32.2565
ABM [27] 19.67 4.9624 0.0349 0.0117 0.9574 30.3146 31.0928
LaBGen-P-Semantic (CV+U) [28] 7.67 3.8878 0.0153 0.0049 0.9788 32.9252 33.5776
LaBGen-P-Semantic (MP+U) [29] 10.00 3.9190 0.0156 0.0051 0.9768 32.8717 33.5317
SPMD [30] 2.50 3.8141 0.0119 0.0021 0.9812 33.9777 34.5761
FSBE [31] 7.33 3.8960 0.0131 0.0028 0.9784 32.7916 33.4660
AAPSA [1] 22.17 5.6500 0.0449 0.0222 0.9575 29.4528 30.1252

Results, for the intermittent motion category.

Click on method name for more details.

Method Average ranking Average     AGE  Average     pEPs  Average   pCEPS  Average   MSSSIM  Average   PSNR     CQM      
Temporal median filter [2] 24.83 6.8003 0.0615 0.0421 0.9150 24.7016 25.7573
Photomontage [3] 24.83 7.1460 0.0639 0.0427 0.9138 24.8941 25.8682
RSL2011 [4] 14.83 5.1116 0.0399 0.0244 0.9532 28.0795 29.0599
FC-FlowNet [5] 19.00 6.7811 0.0599 0.0347 0.9312 27.0272 27.9086
LaBGen [6] 10.67 4.4333 0.0293 0.0164 0.9597 29.4757 30.3373
LaBGen-P [7] 5.33 4.1278 0.0225 0.0115 0.9712 31.9974 32.7260
RMR [8] 5.50 4.3606 0.0213 0.0091 0.9730 31.1372 31.9285
SC-SOBS-C4 [9] 19.17 6.2583 0.0487 0.0238 0.9255 25.9249 26.9569
MAGRPCA [10] 16.17 8.3106 0.1391 0.1018 0.9710 29.7923 30.6718
TMFG [11] 23.67 6.7602 0.0609 0.0409 0.9167 24.8249 25.8626
Bidirectional Analysis and Consensus Voting [12] 6.00 4.5966 0.0198 0.0079 0.9657 30.2170 31.1074
Bidirectional Analysis [13] 11.00 5.0569 0.0245 0.0082 0.9533 28.6439 29.5925
BE-AAPSA [14] 18.00 6.6997 0.0547 0.0369 0.9345 26.5072 27.5251
MSCL [15] 5.17 3.9743 0.0313 0.0215 0.9831 32.6916 33.4541
DECOLOR [16] 25.67 12.0047 0.2000 0.1598 0.9319 26.1289 27.1080
3-Term Decomposition [17] 22.33 9.4001 0.1508 0.1100 0.9338 25.9513 26.9027
BRTF [18] 27.33 10.4812 0.1572 0.1210 0.9315 25.2367 26.2792
RFSA [19] 28.00 10.4876 0.1572 0.1208 0.9313 25.2233 26.2697
RMAMR [20] 26.00 10.3766 0.1566 0.1204 0.9335 25.3660 26.4087
SSGoDec [21] 24.67 10.2804 0.1556 0.1196 0.9358 25.5155 26.5583
GRASTA [22] 23.67 10.2392 0.1553 0.1193 0.9370 25.6080 26.6518
GOSUS [23] 22.67 10.2085 0.1548 0.1189 0.9378 25.6716 26.7113
BEWiS [24] 11.00 4.7798 0.0277 0.0173 0.9585 29.7747 30.6778
LaBGen-OF [25] 7.67 4.6433 0.0221 0.0120 0.9676 30.5799 31.3920
NExBI [26] 9.17 4.6374 0.0248 0.0123 0.9639 30.0986 30.8849
ABM [27] 15.00 5.6742 0.0416 0.0237 0.9550 27.2945 28.3130
LaBGen-P-Semantic (CV+U) [28] 1.33 3.8079 0.0194 0.0082 0.9810 32.8806 33.5830
LaBGen-P-Semantic (MP+U) [29] 3.50 3.9757 0.0206 0.0094 0.9782 32.1411 32.8462
SPMD [30] 4.50 4.1840 0.0206 0.0088 0.9745 31.9703 32.7043
FSBE [31] 12.83 5.3438 0.0399 0.0220 0.9610 29.1968 30.1392
AAPSA [1] 26.50 8.2573 0.0768 0.0522 0.9009 24.2814 25.2087

Results, for the clutter category.

Click on method name for more details.

Method Average ranking Average     AGE  Average     pEPs  Average   pCEPS  Average   MSSSIM  Average   PSNR     CQM      
Temporal median filter [2] 19.00 12.5316 0.1590 0.1108 0.8185 26.1441 27.1507
Photomontage [3] 9.50 6.8195 0.0543 0.0294 0.8892 28.5554 29.4882
RSL2011 [4] 12.33 7.3013 0.0701 0.0375 0.9087 27.9304 28.9763
FC-FlowNet [5] 20.17 12.5556 0.1719 0.1185 0.8762 25.3201 26.4707
LaBGen [6] 14.33 8.0579 0.1035 0.0740 0.8834 26.7690 27.7986
LaBGen-P [7] 12.33 7.8947 0.0986 0.0678 0.8967 28.1140 29.1305
RMR [8] 23.00 15.3119 0.1819 0.1202 0.7306 23.3608 24.5341
SC-SOBS-C4 [9] 11.00 7.0590 0.0644 0.0304 0.8939 28.0077 29.0737
MAGRPCA [10] 11.00 8.1589 0.0647 0.0294 0.9446 26.6872 27.5988
TMFG [11] 19.00 11.7469 0.1397 0.0952 0.8144 25.9395 27.0378
Bidirectional Analysis and Consensus Voting [12] 11.83 7.2284 0.0546 0.0243 0.9206 25.9467 27.0419
Bidirectional Analysis [13] 10.00 6.6565 0.0497 0.0177 0.9243 26.4376 27.5267
BE-AAPSA [14] 21.00 12.3049 0.1775 0.1205 0.8526 23.7151 24.8666
MSCL [15] 2.83 5.2695 0.0275 0.0094 0.9629 31.3743 32.2837
DECOLOR [16] 24.83 16.3016 0.2273 0.1462 0.7792 23.4934 24.4716
3-Term Decomposition [17] 29.00 17.8838 0.2499 0.1714 0.6969 21.7537 22.9806
BRTF [18] 29.50 18.0231 0.2514 0.1645 0.6805 21.4143 22.6510
RFSA [19] 30.50 19.1931 0.2532 0.1696 0.6638 21.1248 22.3588
RMAMR [20] 11.00 8.1589 0.0647 0.0294 0.9446 26.6872 27.5988
SSGoDec [21] 26.17 15.9385 0.2083 0.1340 0.7037 22.1258 23.3488
GRASTA [22] 26.50 16.6606 0.2216 0.1411 0.7045 22.2120 23.4316
GOSUS [23] 26.17 16.4434 0.2153 0.1394 0.7072 22.2092 23.4261
BEWiS [24] 18.00 10.6714 0.1227 0.0845 0.8610 25.4804 26.4783
LaBGen-OF [25] 1.50 4.1821 0.0246 0.0117 0.9640 32.6339 33.4654
NExBI [26] 5.17 5.3091 0.0414 0.0141 0.9379 30.0056 30.9847
ABM [27] 21.17 11.2043 0.1402 0.0829 0.8104 22.8067 23.9437
LaBGen-P-Semantic (CV+U) [28] 8.17 6.0107 0.0578 0.0333 0.9294 29.8567 30.7288
LaBGen-P-Semantic (MP+U) [29] 8.50 6.1222 0.0573 0.0325 0.9290 29.7306 30.6027
SPMD [30] 3.00 4.5998 0.0250 0.0114 0.9572 30.9212 31.8627
FSBE [31] 2.83 4.7660 0.0328 0.0152 0.9641 32.1217 32.9884
AAPSA [1] 25.67 15.7186 0.2577 0.1863 0.8240 22.8853 23.9875

Results, for the jitter category.

Click on method name for more details.

Method Average ranking Average     AGE  Average     pEPs  Average   pCEPS  Average   MSSSIM  Average   PSNR     CQM      
Temporal median filter [2] 2.83 9.0892 0.1063 0.0404 0.8556 25.5526 26.6236
Photomontage [3] 16.00 10.1272 0.1210 0.0441 0.8390 24.3478 25.4186
RSL2011 [4] 22.50 10.5876 0.1237 0.0493 0.8059 22.6947 23.8316
FC-FlowNet [5] 12.83 10.2805 0.1138 0.0446 0.8489 25.0240 26.0705
LaBGen [6] 8.83 9.7096 0.1108 0.0420 0.8487 25.3282 26.3608
LaBGen-P [7] 7.00 9.6487 0.1100 0.0410 0.8504 25.2447 26.2834
RMR [8] 28.33 11.5991 0.1468 0.0624 0.7806 22.1418 23.2998
SC-SOBS-C4 [9] 13.00 10.0232 0.1186 0.0420 0.8403 24.5562 25.6570
MAGRPCA [10] 12.67 10.9525 0.1131 0.0419 0.8503 24.4999 25.6199
TMFG [11] 4.17 9.2013 0.1069 0.0408 0.8543 25.4488 26.5210
Bidirectional Analysis and Consensus Voting [12] 13.33 10.0040 0.1098 0.0365 0.8369 23.5048 24.6902
Bidirectional Analysis [13] 12.50 10.1835 0.1090 0.0341 0.8385 23.6701 24.8364
BE-AAPSA [14] 17.83 10.1994 0.1246 0.0584 0.8373 24.5489 25.6301
MSCL [15] 8.00 9.7403 0.1049 0.0424 0.8475 25.3035 26.3824
DECOLOR [16] 26.67 14.4931 0.2543 0.1459 0.8494 22.4057 23.5875
3-Term Decomposition [17] 24.50 12.5810 0.1405 0.0532 0.8055 22.9781 24.1084
BRTF [18] 27.33 13.3652 0.1558 0.0558 0.8001 22.5448 23.7288
RFSA [19] 26.83 13.7167 0.1640 0.0689 0.7825 22.9790 24.1223
RMAMR [20] 12.67 10.9525 0.1131 0.0419 0.8503 24.4999 25.6199
SSGoDec [21] 22.67 12.4230 0.1428 0.0448 0.8379 22.7464 23.9061
GRASTA [22] 29.83 14.9121 0.2185 0.1241 0.8031 22.3155 23.4959
GOSUS [23] 26.50 13.2675 0.1593 0.0535 0.8210 22.4765 23.6413
BEWiS [24] 5.83 9.4156 0.1048 0.0402 0.8524 24.7408 25.8579
LaBGen-OF [25] 2.17 9.2410 0.1064 0.0380 0.8579 25.9053 26.9264
NExBI [26] 21.67 11.1301 0.1332 0.0513 0.8135 23.2880 24.4286
ABM [27] 24.00 11.6703 0.1390 0.0492 0.8127 22.6091 23.7288
LaBGen-P-Semantic (CV+U) [28] 9.33 9.5900 0.1106 0.0427 0.8491 25.0303 26.0662
LaBGen-P-Semantic (MP+U) [29] 11.17 9.4907 0.1128 0.0435 0.8458 24.7809 25.8366
SPMD [30] 11.00 9.8095 0.1122 0.0397 0.8501 24.3467 25.4432
FSBE [31] 19.33 10.3878 0.1256 0.0530 0.8374 23.9364 25.0050
AAPSA [1] 13.67 10.2185 0.1202 0.0382 0.8545 23.2861 24.2624

Results, for the illumination changes category.

Click on method name for more details.

Method Average ranking Average     AGE  Average     pEPs  Average   pCEPS  Average   MSSSIM  Average   PSNR     CQM      
Temporal median filter [2] 19.00 12.2205 0.2322 0.1783 0.9400 24.3156 25.3760
Photomontage [3] 4.67 5.2668 0.0329 0.0155 0.9743 30.2102 31.0393
RSL2011 [4] 16.83 9.1963 0.0996 0.0669 0.9349 23.9579 25.1714
FC-FlowNet [5] 19.50 13.3662 0.2584 0.1906 0.9452 24.2047 25.2306
LaBGen [6] 6.33 6.1922 0.0440 0.0206 0.9725 29.7108 30.5510
LaBGen-P [7] 11.17 7.4945 0.0611 0.0376 0.9630 25.2155 26.3522
RMR [8] 10.17 7.1869 0.0656 0.0403 0.9485 28.7961 29.8049
SC-SOBS-C4 [9] 15.50 10.3591 0.1005 0.0574 0.9075 26.2190 27.0837
MAGRPCA [10] 9.50 7.7987 0.1158 0.0804 0.9760 31.9554 32.7325
TMFG [11] 29.33 22.0886 0.3025 0.2119 0.8612 20.8670 22.0259
Bidirectional Analysis and Consensus Voting [12] 22.83 16.1236 0.1192 0.0668 0.8653 21.4330 22.5940
Bidirectional Analysis [13] 25.00 16.8302 0.1833 0.0417 0.8252 19.3961 20.6819
BE-AAPSA [14] 11.33 7.0447 0.0694 0.0451 0.9613 27.4897 28.2828
MSCL [15] 2.00 4.4319 0.0341 0.0134 0.9856 34.6735 35.3442
DECOLOR [16] 20.50 15.4263 0.2505 0.2029 0.9423 24.9675 25.8595
3-Term Decomposition [17] 23.83 17.1636 0.2765 0.1965 0.9469 21.9685 23.0674
BRTF [18] 25.83 16.1184 0.2790 0.2146 0.9323 23.0102 24.1362
RFSA [19] 29.00 24.6445 0.3327 0.2648 0.9114 20.9701 22.1231
RMAMR [20] 24.00 15.1097 0.2757 0.2113 0.9407 23.4022 24.5004
SSGoDec [21] 21.17 14.1049 0.2636 0.1983 0.9457 23.7863 24.8676
GRASTA [22] 22.17 14.1049 0.2636 0.1983 0.9457 23.7840 24.8670
GOSUS [23] 20.50 14.1035 0.2636 0.1983 0.9458 23.7883 24.8695
BEWiS [24] 5.67 5.9048 0.0312 0.0223 0.9745 29.5427 30.3805
LaBGen-OF [25] 12.33 8.2200 0.1130 0.0746 0.9654 27.8422 28.7690
NExBI [26] 7.50 4.8310 0.0536 0.0321 0.9621 29.2036 30.1214
ABM [27] 17.33 11.2415 0.1142 0.0866 0.9194 24.8643 25.8980
LaBGen-P-Semantic (CV+U) [28] 23.33 14.1797 0.1633 0.1215 0.8853 21.2098 22.3803
LaBGen-P-Semantic (MP+U) [29] 26.50 18.5133 0.2076 0.1565 0.8542 20.4792 21.6238
SPMD [30] 2.50 4.4750 0.0222 0.0090 0.9835 32.1311 32.9629
FSBE [31] 2.17 5.5089 0.0150 0.0040 0.9854 32.1672 32.9817
AAPSA [1] 8.33 6.7259 0.0562 0.0250 0.9728 28.5080 29.2912

Results, for the background motion category.

Click on method name for more details.

Method Average ranking Average     AGE  Average     pEPs  Average   pCEPS  Average   MSSSIM  Average   PSNR     CQM      
Temporal median filter [2] 8.17 9.6479 0.1262 0.0300 0.8566 25.9277 26.7931
Photomontage [3] 27.00 12.0930 0.1589 0.0410 0.8244 23.5420 24.5253
RSL2011 [4] 28.50 13.2090 0.1604 0.0531 0.8040 23.8834 24.7017
FC-FlowNet [5] 11.00 10.0539 0.1440 0.0362 0.8612 26.0302 26.9014
LaBGen [6] 14.33 10.4996 0.1356 0.0334 0.8465 25.4982 26.1616
LaBGen-P [7] 14.17 10.5858 0.1350 0.0336 0.8442 25.7879 26.5748
RMR [8] 27.83 12.2932 0.1682 0.0464 0.8151 23.9116 24.7040
SC-SOBS-C4 [9] 16.33 10.7280 0.1481 0.0302 0.8486 24.5806 25.5603
MAGRPCA [10] 9.00 10.0742 0.1318 0.0299 0.8748 25.8756 26.7832
TMFG [11] 3.50 9.0773 0.1185 0.0228 0.8706 26.4052 27.2770
Bidirectional Analysis and Consensus Voting [12] 11.67 10.5260 0.1316 0.0269 0.8553 25.2106 26.1305
Bidirectional Analysis [13] 15.17 10.7772 0.1350 0.0273 0.8574 24.0802 25.0203
BE-AAPSA [14] 5.17 9.3755 0.1266 0.0259 0.8766 26.0041 26.9062
MSCL [15] 19.33 11.2194 0.1540 0.0332 0.8448 24.4813 25.6982
DECOLOR [16] 17.67 10.5910 0.1403 0.0351 0.8535 24.2455 25.1072
3-Term Decomposition [17] 16.17 10.6576 0.1409 0.0336 0.8564 24.5778 25.4719
BRTF [18] 26.00 11.7599 0.1532 0.0423 0.8335 23.5254 24.5149
RFSA [19] 24.50 11.7422 0.1527 0.0421 0.8338 23.5308 24.5188
RMAMR [20] 27.83 11.7876 0.1539 0.0428 0.8331 23.5139 24.5008
SSGoDec [21] 24.83 11.7587 0.1532 0.0430 0.8329 23.5442 24.5264
GRASTA [22] 26.50 11.7686 0.1536 0.0431 0.8326 23.5367 24.5196
GOSUS [23] 28.00 11.7763 0.1536 0.0432 0.8322 23.5296 24.5127
BEWiS [24] 5.83 9.6776 0.1258 0.0286 0.8644 26.0753 26.9685
LaBGen-OF [25] 10.83 10.0698 0.1312 0.0323 0.8550 25.8626 26.6974
NExBI [26] 22.17 11.5851 0.1577 0.0379 0.8347 24.3680 25.2364
ABM [27] 6.67 9.9952 0.1316 0.0292 0.8662 26.3616 27.5133
LaBGen-P-Semantic (CV+U) [28] 2.00 8.8557 0.1165 0.0202 0.8797 26.9364 28.0108
LaBGen-P-Semantic (MP+U) [29] 1.00 8.7583 0.1152 0.0189 0.8805 27.0334 28.1119
SPMD [30] 9.00 9.9115 0.1252 0.0289 0.8587 25.5415 26.2744
FSBE [31] 15.83 10.5862 0.1447 0.0370 0.8460 25.6187 26.4072
AAPSA [1] 20.00 11.1404 0.1488 0.0381 0.8422 24.4876 25.4679

Results, for the very long category.

Click on method name for more details.

Method Average ranking Average     AGE  Average     pEPs  Average   pCEPS  Average   MSSSIM  Average   PSNR     CQM      
Temporal median filter [2] 12.50 6.9588 0.0557 0.0199 0.9843 29.3160 30.2107
Photomontage [3] 14.33 6.6446 0.0629 0.0259 0.9838 29.2081 30.0166
RSL2011 [4] 30.67 13.2990 0.1980 0.1170 0.8544 23.9750 25.0400
FC-FlowNet [5] 18.00 7.7727 0.0673 0.0189 0.9690 28.6840 29.5294
LaBGen [6] 10.83 5.5689 0.0327 0.0075 0.9807 29.2590 30.1356
LaBGen-P [7] 19.83 7.3028 0.0803 0.0395 0.9760 28.2974 29.1409
RMR [8] 30.33 13.2459 0.2547 0.1571 0.9115 25.1309 26.0738
SC-SOBS-C4 [9] 10.00 6.0638 0.0355 0.0021 0.9837 29.2615 30.1014
MAGRPCA [10] 5.00 3.9164 0.0184 0.0040 0.9861 31.3101 32.0265
TMFG [11] 15.17 7.3530 0.0671 0.0274 0.9851 29.2432 30.1213
Bidirectional Analysis and Consensus Voting [12] 17.33 7.0049 0.0429 0.0045 0.9558 27.6839 28.6442
Bidirectional Analysis [13] 14.17 6.7502 0.0395 0.0012 0.9675 27.8260 28.7710
BE-AAPSA [14] 3.00 3.8745 0.0148 0.0010 0.9844 32.5501 33.1853
MSCL [15] 3.33 3.8214 0.0172 0.0022 0.9874 32.2773 32.9941
DECOLOR [16] 14.83 5.5234 0.0352 0.0155 0.9524 27.8960 28.7663
3-Term Decomposition [17] 24.33 7.6075 0.1108 0.0591 0.9520 27.7971 28.6593
BRTF [18] 24.00 7.3847 0.0764 0.0287 0.9515 27.6387 28.5167
RFSA [19] 26.00 7.5854 0.0913 0.0389 0.9510 27.5835 28.4562
RMAMR [20] 5.00 3.9164 0.0184 0.0040 0.9861 31.3101 32.0265
SSGoDec [21] 22.00 7.3409 0.0745 0.0276 0.9515 27.6654 28.5486
GRASTA [22] 22.00 7.3212 0.0724 0.0269 0.9510 27.6568 28.5354
GOSUS [23] 23.50 7.3589 0.0747 0.0276 0.9514 27.6554 28.5304
BEWiS [24] 2.00 3.9652 0.0108 0.0006 0.9891 32.5325 33.2217
LaBGen-OF [25] 3.17 4.2856 0.0114 0.0006 0.9891 32.0746 32.8312
NExBI [26] 21.67 6.2698 0.0597 0.0172 0.9393 26.7234 27.6388
ABM [27] 8.17 4.8410 0.0242 0.0023 0.9784 30.3236 31.0717
LaBGen-P-Semantic (CV+U) [28] 25.33 8.2348 0.1011 0.0488 0.9477 27.7941 28.6867
LaBGen-P-Semantic (MP+U) [29] 23.83 8.2884 0.1001 0.0429 0.9440 28.4847 29.3482
SPMD [30] 9.33 6.0926 0.0283 0.0055 0.9824 30.3175 31.1195
FSBE [31] 16.33 6.9832 0.0780 0.0352 0.9783 29.3262 30.1744
AAPSA [1] 19.00 6.6297 0.0547 0.0129 0.9614 27.4929 28.3462

Results, for the very short category.

Click on method name for more details.

Method Average ranking Average     AGE  Average     pEPs  Average   pCEPS  Average   MSSSIM  Average   PSNR     CQM      
Temporal median filter [2] 5.33 5.1336 0.0324 0.0120 0.9537 30.6255 31.3012
Photomontage [3] 3.67 4.9770 0.0327 0.0030 0.9548 31.0117 31.6568
RSL2011 [4] 26.83 9.0950 0.0877 0.0410 0.8859 24.2435 25.1426
FC-FlowNet [5] 17.33 6.5094 0.0541 0.0211 0.9345 28.9447 29.6743
LaBGen [6] 10.83 5.3093 0.0332 0.0136 0.9474 30.1754 30.8162
LaBGen-P [7] 13.17 5.5653 0.0417 0.0199 0.9461 29.8264 30.4694
RMR [8] 18.00 6.4059 0.0541 0.0137 0.9329 28.0882 28.8900
SC-SOBS-C4 [9] 5.83 5.2953 0.0330 0.0044 0.9556 30.4997 31.1813
MAGRPCA [10] 20.33 8.3363 0.0999 0.0778 0.9511 27.8815 28.6430
TMFG [11] 5.50 5.1825 0.0321 0.0106 0.9520 30.6010 31.2242
Bidirectional Analysis and Consensus Voting [12] 26.17 9.0271 0.0850 0.0366 0.8902 22.8906 23.8972
Bidirectional Analysis [13] 19.50 6.3972 0.0480 0.0126 0.9279 27.0145 27.8472
BE-AAPSA [14] 24.17 8.0857 0.0832 0.0429 0.8921 25.6458 26.5128
MSCL [15] 10.17 5.7790 0.0387 0.0131 0.9363 31.2396 31.8320
DECOLOR [16] 26.17 8.9984 0.1058 0.0809 0.9475 27.5064 28.2282
3-Term Decomposition [17] 19.17 7.7930 0.1019 0.0780 0.9506 29.2653 29.9966
BRTF [18] 24.67 8.7119 0.1039 0.0785 0.9477 27.5693 28.2903
RFSA [19] 24.33 8.8522 0.1042 0.0769 0.9441 27.5926 28.3072
RMAMR [20] 20.33 8.3363 0.0999 0.0778 0.9511 27.8815 28.6430
SSGoDec [21] 20.50 8.2366 0.0989 0.0774 0.9510 27.8814 28.6231
GRASTA [22] 20.17 8.1496 0.0995 0.0779 0.9509 28.0017 28.7774
GOSUS [23] 20.33 7.9443 0.0998 0.0786 0.9506 28.5067 29.2419
BEWiS [24] 10.67 5.1937 0.0345 0.0136 0.9488 29.8036 30.4858
LaBGen-OF [25] 7.83 5.0338 0.0325 0.0135 0.9509 30.4959 31.1315
NExBI [26] 14.17 5.7134 0.0427 0.0106 0.9424 28.7691 29.4964
ABM [27] 22.50 7.7572 0.0664 0.0238 0.9101 26.2541 27.0108
LaBGen-P-Semantic (CV+U) [28] 1.50 4.5450 0.0245 0.0056 0.9629 31.4073 32.0297
LaBGen-P-Semantic (MP+U) [29] 2.83 4.7175 0.0273 0.0067 0.9612 31.0652 31.6927
SPMD [30] 15.00 5.9017 0.0446 0.0177 0.9423 29.5449 30.2557
FSBE [31] 10.33 5.4912 0.0347 0.0042 0.9477 29.5437 30.2593
AAPSA [1] 27.67 9.2952 0.0860 0.0438 0.8870 22.7636 23.7275

Metrics :

  • AGE: (Average Gray-level Error). Average of the gray-level absolute difference between GT and the computed background (CB) image.
  • pEPs: (Percentage of Error Pixels). Percentage of EPs (number of pixels in CB whose value differs from the value of the corresponding pixel in GT by more than a threshold) with respect to the total number of pixels in the image.
  • pCEPS: (Percentage of Clustered Error Pixels). Percentage of CEPs (number of pixels whose 4-connected neighbors are also error pixels) with respect to the total number of pixels in the image.
  • MSSSIM: (MultiScale Structural Similarity Index). Estimate of the perceived visual distortion.
  • PSNR: (Peak-Signal-to-Noise-Ratio) Amounts to 10log_10((L-1)^2/MSE) where L is the maximum number of grey levels and MSE is the Mean Squared Error between GT and CB images.
  • CQM: (Color image Quality Measure). Based on a reversible transformation of the YUV color space and on the PSNR computed in the single YUV bands. It assumes values in db and the higher the CQM value, the better is the background estimate.


References :

  • 1. M Chacon-Murguia, G Ramirez-Alonso and J Ramirez-Quintana "Evaluation of the Background Modeling Method Auto-Adaptive Parallel Neural Network Architecture in the SBMnet Dataset", Scene Background Modeling workshop (ICPR) 2016
  • 2. Piccardi, Massimo. "Background subtraction techniques: a review." In Systems, man and cybernetics, 2004 IEEE international conference on, vol. 4, pp. 3099-3104. IEEE, 2004.
  • 3. A. Agarwala, M. Dontcheva, M. Agrawala, S. Drucker, A. Colburn, B. Curless, D. Salesin, and M. Cohen, “Interactive digital photomontage,” ACM Trans. Graph., vol. 23, no. 3, pp. 294–302, Aug. 2004
  • 4. V. Reddy, C. Sanderson, and B. C. Lovell, “A low-complexity algorithm for static background estimation from cluttered image sequences in surveillance contexts,” EURASIP J. Image Video Process., vol. 2011, pp. 1:14, Jan. 2011
  • 5. I. Halfaoui, F. Bouzaraa and O. Urfalioglu. "CNN-Based Initial Background Estimation", Scene Background Modeling workshop (ICPR) 2016
  • 6. B. Laugraud, S. Piérard, M. Van Droogenbroeck « LaBGen: A method based on motion detection for generating the background of a scene », Pattern Recognition Letters, Volume 96, Pages 12-21, 1 September 2017
  • 7. B. Laugraud, S. Piérard, M. Van Droogenbroeck "LaBGen-P: A Pixel-Level Stationary Background Generation Method Based on LaBGen", Scene Background Modeling workshop (ICPR) 2016
  • 8. D. Ortego, J. C. SanMiguel, J. M. Martínez, "Rejection based multipath reconstruction for background estimation in video sequences with stationary objects", Computer Vision and Image Understanding, vol. 147, pp. 23-37, 2016
  • 9. L. Maddalena and A.Petrosino "Extracting a Background Image by a Multi-modal Scene Background Model", Scene Background Modeling workshop (ICPR) 2016
  • 10. S Javed, A. Mahmmod, T. Bouwmans and S. Ki Jung. "Motion-Aware Graph Regularized RPCA for Background Modeling of Complex Scene", Scene Background Modeling workshop (ICPR) 2016
  • 11. W. Liu, Y. Cai, M. Zhang, H. Li and H. Gu. "Scene Background Estimation Based on Temporal Median Filter with Gaussian Filtering", Scene Background Modeling workshop (ICPR) 2016
  • 12. T. Minematsu, A. Shimada and R-I Taniguchi "Background Initialization based on Bidirectional Analysis and Consensus Voting", , Scene Background Modeling workshop (ICPR) 2016
  • 13. T. Minematsu, A. Shimada and R-I Taniguchi "Background Initialization based on Bidirectional Analysis and Consensus Voting", , Scene Background Modeling workshop (ICPR) 2016
  • 14. G. Ramirez-Alonsoa, Juan A.Ramirez-Quintana .Mario I.Chacon-Murguia "Temporal weighted learning model for background estimation with an automatic re-initialization stage and adaptive parameters update", in Pattern Recognition Letters, 2017 (http://www.sciencedirect.com/science/article/pii/S0167865517300120)
  • 15. Sajid Javed, Arif Mahmood, Thierry Bouwmans, and Soon Ki Jung, "Background-Foreground Modeling Based on Spatiotemporal Sparse Subspace Clustering" accepted at IEEE Trans on Image Processing, 2017
  • 16. X. Zhou, C. Yang, W. Yu, Moving Object Detection by Detecting Contiguous Outliers in the Low-Rank Representation, T-PAMI-2013
  • 17. Omar Oreifej, Xin Li, Mubarak Shah, "Simultaneous Video Stabilization and Moving Object Detection in Turbulence", T-PAMI, 2012
  • 18. Q. Zhou etal., Bayesian Robust Tensor Factorization for Incomplete Multiway Data, IEEE Transactions on NNLS, 2015
  • 19. X. Guo et al., Robust Foreground Detection Using Smoothness and Arbitrariness Constraints, ECCV-2014
  • 20. X. Ye, Foreground-Background Separation From Video Clips via Motion-Assisted Matrix Restoration, T-CSVT, 2015 => https://github.com/andrewssobral/lrslibrary
  • 21. T. Zhaou et al., GoDec: Randomized Low-rank & Sparse Matrix Decomposition in Noisy Case,ICML2011
  • 22. Jun He et al. Incremental Gradient on the Grassmannian for Online Foreground and Background Separation in Subsampled Video, CVPR 2012
  • 23. J. Xu et al., GOSUS: Grassmannian Online Subspace Updates with Structured-sparsity, ICCV-2013
  • 24. Massimo De Gregorio and Maurizio Giordano, "Background estimation by weightless neural networks” - submitted to Pattern Recognition Letters
  • 25. B. Laugraud, M. Van Droogenbroeck « Is a Memoryless Motion Detection Truly Relevant for Background Generation with LaBGen? », Advanced Concepts for Intelligent Vision Systems (ACIVS), LNCS 10617, Pages 443-454, 2017
  • 26. M.Jmal "Real-time Scene Background Initialization based on Spatio-Temporal Neighborhood Exploration" submitted to journal of multimedia tools and applications.
  • 27. Danilo Avola, Marco Bernardi, Luigi Cinque, Gian Luca Foresti, Cristiano Massaroni "Adaptive bootstrapping management by keypoint clustering for background initialization", Pattern Recognition Letters Volume 100, 1 December 2017, Pages 110-116 (www.sciencedirect.com/science/article/pii/S0167865517303975)
  • 28. B. Laugraud, S. Piérard, M. Van Droogenbroeck « LaBGen-P-Semantic: A First Step for Leveraging Semantic Segmentation in Background Generation », MDPI Journal of Imaging, Volume 4, Issue 7, 25 June 2018 (http://www.mdpi.com/2313-433X/4/7/86)
  • 29. B. Laugraud, S. Piérard, M. Van Droogenbroeck « LaBGen-P-Semantic: A First Step for Leveraging Semantic Segmentation in Background Generation », MDPI Journal of Imaging, Volume 4, Issue 7, 25 June 2018 (http://www.mdpi.com/2313-433X/4/7/86)
  • 30. Z. Xu, B. Min, R. Cheung "A Robust Background Initialization Algorithm with Superpixel Motion Detection". submitted to Signal Processing: Image Communication; arxiv.org/abs/1805.06737
  • 31. Achraf Djerida Robust background generation based on an effective frames selection method and an efficient background estimation procedure (FSBE) Journal Signal Processing: Image Communication DOI 10.1016/j.image.2019.06.001