8/22/2017 - 6:19 AM

GSoC 2017 - Improve Background Subtraction with Aggregated Saliency

GSoC 2017 - Improve Background Subtraction with Aggregated Saliency

Improve Background Subtraction with Aggregated Saliency

Contributor: Shengxin Qian | Mentor: Antonella Cascitelli

1. Overview

This project contains the implementation of three saliency algorithms:

  • DeepGaze1 [1] Image based saliency

    • This is a supervised learning algorithm based on the linear combination of AlexNet 5th convolution layers.
    • This algorithm takes advantage of the high level features generated by those pretrained Deep Neural Network.
    • The training of this algorithm is slow but it is fast when estimating saliency.
    • This algorithm can achieve 0.84 AUC on MIT Saliency Benchmark which is one of the best score.
    • This algorithm can be retrained with cutting edge Deep Neural Network and achieve much better performance.
  • Discriminant Saliency [2] Video based saliency

    • This is an unsupervised learning algorithm based on dynamic texture model and “center surround” local feature contrast.
    • This algorithm exhibits substantial robustness, performing well in the presence of camera motion, variable foreground object scales, and low imaging quality
    • The drawback of this algorithm is its the processing speed. Depending on the output resolution, each frame takes from several seconds to more than a hundred seconds. However, it has potential to be highly scalable if you can slightly modify it to parallel version because the processing of each sliding window is independent.
  • Background Contrast [3] Image based saliency

    • This is an unsupervised learning algorithm based on superpixels' boundary connectivity and region feature contrast.
    • This algorithm is able to estimate the probability of being background objects and be effective on those images that saliency objects occupy certain amount of space.
    • The downside of this algorithm is it can not handle those images when high level cues is critical or the scene is complex.
    • This module offers the opportunity of optimizing the result of other saliency algorithms and segmenting those saliency objects.

List of commits during GSoC 2017


  1. Kümmerer, Matthias, Lucas Theis, and Matthias Bethge. "Deep gaze i: Boosting saliency prediction with feature maps trained on imagenet." arXiv preprint arXiv:1411.1045 (2014).
  2. Mahadevan, Vijay, and Nuno Vasconcelos. "Spatiotemporal saliency in dynamic scenes." IEEE transactions on pattern analysis and machine intelligence 32.1 (2010): 171-177.
  3. Zhu, Wangjiang, et al. "Saliency optimization from robust background detection." Proceedings of the IEEE conference on computer vision and pattern recognition . 2014.