The Secret of Bokeh

Bokeh, the technique of deliberately creating out-of-focus area in photos, has been widely used by photographers for emphasizing semantic subjects. These subjects could be either foreground or background of photos. An example of portrait bokeh is shown below:

Photograph by Di Liu

The effect of bokeh is usually introduced by particular lens in SLR cameras. However, camera in mobile devices has become more and more ubiquitous and also a common choice to record daily life nowadays. So the question arises that how can we design algorithms to generate the effect of bokeh in well-focused photo shot by mobile devices.

In fact, this functionality (creating semantically out-of-focus photo from well-focused photo) is a well-studied topic in computational photography. And a typical algorithm comprises two stages: depth map estimation and selective blurring.

We would like to obtain depth map of the underlying photo in the first place to distinguish between foreground and background. But estimating depth map from a single image has long been known as a difficult task. A practical solution is to leverage information in image sequence (burst mode in mobile devices) so that a multi-view stereo could be formed. Google's Lens Blur adopts this kind of practice and they had a research blog to elaborate detailed techniques. BTW, burst images could also be utilized for denoising. After that, we can perform adaptive per-pixel blurring according to the computed depth map. Or more advanced technique could be adopted to enable motion blur rendering. To make the final result more seamless and natural, the above rendering process could be done in a multi-scale manner, e.g. local Laplace pyramids. The same pipeline could also be applied to generate 3D effect image from burst images, as demonstrated in this website.