+2  A: 

You probably want to fit to closed bezier splines (curves).

There is a page with a Java applet that does such fitting. Take a look at their (uncommented, bleh) PSegment.java

If that doesn't do the job well enough, Fitting B-Spline Curves to Point Clouds by Squared Distance Minimization should be overkill for the task.

For more info, Google (Scholar) searches for things like "b-spline fitting" and "closed bezier spline" should help.

Jay Kominek
+1  A: 

MATLAB does this with a contour plot: see the contour function -- it produces contour line data from a grid of f(x,y) data. I don't know much about the algorithm they use (they do have a page which discusses it) but maybe worth looking at the Octave equivalent.

there's a thesis on contour plotting and a CodeProject page based on the thesis.

Jason S
A: 

It's the same as converting any other raster object into a vector object, only with another dimension.

Think about how this works with 2D images (do some research) then extrapolate the techniques into 3D.

Adam Davis