iphone camerOverlay for use with Alternate Reality applications

Question

Does anyone know a way to take an image captured on the iphone's camera, and do some image processing (e.g. edge detection, skeletization), and then overlay parts of the processed image on the original image (e.g. only the highlighted edges).

More generically how do I create a UImage with transparency (do I just scale the image and overlay it with an alpha value, does UIImage support transparency like gifs do). I'm thinking that you could combine a UIImagePickerController with a background thread that takes "screenshots" of the UIImagePickerController view and does image processing on it to detect various objects and provide an overlay augmented reality display.

There's an open source simple image processing library for the iphone. The demo shows an example of taking an original photo (of a sudoku board) and then overlaying the detected object in the original photo.

They explain some of the high-level techniques on their blog .

Answer 1

OpenCV makes image overlays remarkably simple, and it's been ported to the iPhone. With OpenCV you can choose to take screenshots as you considered or do image processing on a live stream, one frame at a time. Take a look at some of its tutorial programs, they're really helpful.

Answer 2

I've just done a blog post on how to this exact thing:

http://cmgresearch.blogspot.com/2010/01/augmented-reality-on-iphone-how-to_01.html

Taking screen shots is relatively trivial now since Apple announced that we were allowed to use the UIGetScreenImage functions. In OS 3.1 they also opened up the camera picker interface so you can overlay custom views on the camera.

There's a sample project attached to the blog post.

Answer 3

There's an UIImagePickerController.overlayView property you might want to set:

// Create a new image picker instance:
UIImagePickerController *picker = [[UIImagePickerController alloc] init];

// Set the image picker source:
picker.sourceType = UIImagePickerControllerSourceTypeCamera;

// Hide the controls:
picker.showsCameraControls = NO;
picker.navigationBarHidden = YES;

// Make camera view full screen:
picker.wantsFullScreenLayout = YES;
picker.cameraViewTransform = CGAffineTransformScale(picker.cameraViewTransform, 1, 1.12412);

// Create an overlay view
// this might need to either 1) be transparent, or 2) be of the other dimensions
OverlayView *overlay = [[OverlayView alloc] initWithFrame:CGRectMake(0, 0, self.view.width, self.view.height)];

// Insert the overlay:
picker.cameraOverlayView = overlay;