Have you ever wanted to turn a black and white photo to color, or perhaps you’ve decided that you have better fashion sense than an artist, and you’d like to alter the colors in a painting. Researchers at the Hebrew University of Jerusalem have came up with a simple method that allows you to do just that.
Original Painting:

Painting with Color Changes:

This is certainly not a new problem, and in the past artists have recolored images by hand or with the help of computers, but regardless the process was fairly time demanding. This new method uses much simpler controls, and strikingly simple concept but works quite well. Users mark images with the color of choice using a paint brush tool reminiscent of paint. Then a Matlab program applies that color to neighboring pixels using the rule that neighoboring pixels of similar intensities should be similar colors. Thus boundaries are created within the images, and based on the drawn in colors the image is colorized.

From left to right the above images are the black and white image (with color markings), the colorized version, and the original version. The colorized version is fairly impressive, matching the original image and most aspects. There are a few notable differences: the stain on the boys shirt and the coloration near his hairline. However, it seems likely that a more skilled user might be able to refine the original markings for better results.

Based on past work, they use the function U(r)=aY(r)+b to determine pixel color, where U(r) is the color, Y(r) is the intensity, and a and b are coefficients related to the neighbors of pixel r. This assumes a linear relation between color and intensity, which the work of Zomet and Peleg in 2002 demonstrated. With the addition of weighting functions and a function that determines whether two pixels are neighbors, a sparse system of equations is created. This system is enormous due to the large number of pixels and constraints upon the system of equations. In order to solve these constraints they used Matlabs built in least square solver.

Marked up Black and White Photo:

Colorized Output

The results are shocking considering the amount of detail in the second image compared to the first.

Similar work on moving images are also described in the paper, with the primary difference being that the algorithm adds an additional element, movement, and uses this attribute to calculated neighbors between frames. When colorizing a video clip intermittent frames are marked; in one example 7 frames out of 83 are marked. The results are impressive based on the simplicity of the concept, but look far less realistic than the stills.

You can experiment with the application provided that you have Matlab by downloading there code here:code. note: it took me a bit of work to get it running properly, but in the end I was able to convert a black and white image to color, but my results were less spectacular, though its seems probable that more artistic talent on my part would have helped significantly.

The paper published by Levin, Lischinski, and Weiss can be found here.
http://www.cs.huji.ac.il/~yweiss/Colorization/colorization-siggraph04.pdf