Can we identify a photo in a photo?

Question

I was browsing through some photos uploaded by a friend on Orkut [Orkut has this new feature to tell you how many unnamed people are there in the photo]. There was this particular photo in which there was an advertisement poster with a photo of a person. Orkut displayed that there are two unnamed persons in the photo. Out of curiousity, I just want to know whether it is possible to identify that there is a photo in a photo? If not, can you think of a way that may help an application to identify a photo in a photo?

Answer 1

I would say that this is a difficult problem.

What is the difference between a person in a photo, and a person looking through a window frame?

The software would have to look for differences in lighting and differences in perspective in the photo, but even this could be due to actual dynamics in the photo, such as off-scene lights, or a mirror.

My guess is that any solution would only work some of the time. Just my 2 cents.

Answer 2

You might be able to create a neural network to identify "photos" contained within images.

A quick google search came up with this code project article on image recognition using neural networks, in case you feel like coding it ;)

EDIT

You can use this NN in conjunction with however you plan on identifying people in photos. If a "photo" overlaps a person, then that person is in the photo.

Answer 3

It seems like it would be, although I'm not aware of any specific techniques for doing this. Off the cuff, an easy way to tell for general scenes would be to look at the luminance gradient for the picture. If there's a significant edge, it's probably an image that doesn't really belong to the rest of the scene.

You can think about how this would work intuitively: the lighting that hits a scene will come from the lights in the scene, but the lighting in a photo inside the scene has already been set when the picture was taken. Thus it will probably conflict with the lighting in the scene, and voila, you have a difference that can be identified.

However, specifically identifying a photo (as opposed to a billboard, a sign on a truck, a television, etc.) seems like it would definitely be a challenge.

Answer 4

If you have a photo and there are 2 people in it, whether they are both real or one is in a poster, then don't you have 2 unnamed people if neither have been named before?

In this case as long as the poster can be identified as being a person then it is true that there are 2 undidentified people 'the photo' in question. Right?

Remember, the photo is an entity, not a collection of entities, with differing rules.

Answer 5

Any object that appears in a photo inside a photo will have the wrong shadows. Assuming you identified the fact that it's a face, you can construct a 3D model. If you have part of a picture, inscribed in a rectangle, which does not have the shading of the rest of the picture, it's a picture within a picture.

Answer 6

I am pretty sure that if any solution exists, the guys behind this would have used it.

http://www.boingboing.net/2008/06/29/japanese-cigmachines.html

Answer 7

One handy option regarding posters would be checking the glossiness of each person; A poster would typically be more glossy than other parts of the image.