The Shape Boltzmann Machine: a Strong Model of Object Shape

11 Oct 2013

A good model of object shape is essential in applications such as segmentation, object detection, inpainting and graphics. For example, when performing segmentation, local constraints on the shape can help where the object boundary is noisy or unclear, and global constraints can resolve ambiguities where background clutter looks similar to part of the object. In general, the stronger the model of shape, the more performance is improved.

In this paper, we use a type of Deep Boltzmann Machine that we call a Shape Boltzmann Machine (ShapeBM) for the task of modeling binary shape images. We show that the ShapeBM characterizes a strong model of shape, in that samples from the model look realistic and it can generalize to generate samples that differ from training examples. We find that the ShapeBM learns distributions that are qualitatively and quantitatively better than existing models for this task.

The ShapeBM’s ability to do shape completion suggests applications in a computer graphics setting. Sampled completions can be constrained in real-time by simply clamping certain pixels of the image. The model’s ability to generalize enables it to generate samples that satisfy the user’s constraints. The model’s accurate knowledge about object shape ensures that the samples remain realistic.

Downloads

IJCV link, IJCV preprint PDF, CVPR PDF, GUI ZIP

Video

Stream, Small M4V (4.7 MB), Large M4V (5.6 MB)

BibTex

@inproceedings{Eslami2013, 
    author = {S. M. Ali Eslami and Nicolas Heess and Christopher K. I. Williams and John Winn}, 
    title = {The Shape Boltzmann Machine: a Strong Model of Object Shape},
    booktitle = {International Journal of Computer Vision}, 
    year = {2013}
}

@inproceedings{Eslami2012, 
    author = {S. M. Ali Eslami and Nicolas Heess and John Winn}, 
    title = {The Shape Boltzmann Machine: a Strong Model of Object Shape},
    booktitle = {IEEE Conference on Computer Vision and Pattern Recognition 2012}, 
    year = {2012}
}