Context: The Leading Cue from Cancer Formation to Computer Vision

Recently, I watched a TED lecture named "Experiments that point to a new understanding of cancer" by Mina Bissell, in which the enthusiastic geneticist talked about how context and architecture influence the formation of cancer cells instead of the traditional view of nuclei mutation. This new perspective in turn provokes me to think about the role context plays in understanding what we see, that is, how context becomes a leading cue in computer vision.

Below is the TED lecture for those who may be interested. And the source URL is: http://www.ted.com/talks/mina_bissell_experiments_that_point_to_a_new_understanding_of_cancer.html . Here, I will mainly present how we can utilize context cues in the computer vision applications from three levels: the feature level, the image level and the dataset level. Some of my own experimental results using these approaches are shown as well. Let's get started:

1. The Feature Level (Shape Context):

Shape Context (SC) is a well-known shape descriptor that is proposed by Serge Belongie and Jitendra Malik. The core idea of Shape Context is that the shape of a certain object can be described by the relative positions of other points on the contour to a given point on the contour. And the details of this algorithm can be found at http://www.eecs.berkeley.edu/Research/Projects/CS/vision/shape/sc_digits.html. Below are some preliminary experimental results using this SC method:

Then the 2-D histogram is used as the portrait of context between points on the object contour.

2. The Image Level (Context Aware Saliency):

Visual saliency is where your attention goes when you are looking at an image. Context Aware Saliency argues that it is not only the saliency region, but also the context around it, that helps us to annotate an image accurately. The context aware saliency is implemented by adding exponential decayed weights around the fixed saliency region in a saliency map. And the details of this algorithm can be found at http://cgm.technion.ac.il/Computer-Graphics-Multimedia/Software/Saliency/Saliency.html. Some of my experimental results are shown below:

The saliency map generated by the Context Aware method includes both me and the Batman movie poster into "interesting region", which convey the true meaning of this photo ------ I went to see the Batman movie! In contrast, the saliency map generated by frequency-tuned method is also presented below:

3. The Dataset Level (Transfer Learning):

Transfer learning is what happens when someone finds it much easier to learn to play chess having already learned to play checkers; or to recognize tables having already learned to recognize chairs; or to learn Spanish having already learned Italian. Achieving significant levels of transfer learning across tasks -- that is, achieving cumulative learning -- is perhaps the central problem facing machine learning.

In a word, transfer learning is how to utilize all the context knowledge around you (like all the images in a given dataset) to address an unknown problem. (like classification) in an unsupervised manner. There is an interesting competition named "Unsupervised and Transfer Learning Challenge". Despite that the competition is over, you can still try it and enjoy the beautiful side of transfer learning.