Projecting a 3D point to a 2D screen coordinate

Question

Based on information in Chapter 7 of 3D Programming For Windows (Charles Petzold), I've attempted to write as helper function that projects a Point3D to a standard 2D Point that contains the corresponding screen coordinates (x,y):

public Point Point3DToScreen2D(Point3D point3D,Viewport3D viewPort )
{
    double screenX = 0d, screenY = 0d;

    // Camera is defined in XAML as:
    //        <Viewport3D.Camera>
    //             <PerspectiveCamera Position="0,0,800" LookDirection="0,0,-1" />
    //        </Viewport3D.Camera>

    PerspectiveCamera cam = viewPort.Camera as PerspectiveCamera;

    // Translate input point using camera position
    double inputX = point3D.X - cam.Position.X;
    double inputY = point3D.Y - cam.Position.Y;
    double inputZ = point3D.Z - cam.Position.Z;

    double aspectRatio = viewPort.ActualWidth / viewPort.ActualHeight;

    // Apply projection to X and Y
    screenX = inputX / (-inputZ * Math.Tan(cam.FieldOfView / 2));

    screenY = (inputY * aspectRatio) / (-inputZ * Math.Tan(cam.FieldOfView / 2));

    // Convert to screen coordinates
    screenX = screenX * viewPort.ActualWidth;

    screenY = screenY * viewPort.ActualHeight;


    // Additional, currently unused, projection scaling factors
    /*
    double xScale = 1 / Math.Tan(Math.PI * cam.FieldOfView / 360);
    double yScale = aspectRatio * xScale;

    double zFar = cam.FarPlaneDistance;
    double zNear = cam.NearPlaneDistance;

    double zScale = zFar == Double.PositiveInfinity ? -1 : zFar / (zNear - zFar);
    double zOffset = zNear * zScale;

    */

    return new Point(screenX, screenY);
}

On testing however this function returns incorrect screen coordinates (checked by comparing 2D mouse coordinates against a simple 3D shape). Due to my lack of 3D programming experience I am confused as to why.

The block commented section contains scaling calculations that may be essential, however I am not sure how, and the book continues with the MatrixCamera using XAML. Initially I just want to get a basic calculation working regardless of how inefficient it may be compared to Matrices.

Can anyone advise what needs to be added or changed?

Answer 1

1. It's not clear what you are trying to achieve with aspectRatio coeff. If the point is on the edge of field of view, then it should be on the edge of screen, but if aspectRatio!=1 it isn't. Try setting aspectRatio=1 and make window square. Are the coordinates still incorrect?

2. ActualWidth and ActualHeight seem to be half of the window size really, so screenX will be [-ActualWidth; ActualWidth], but not [0; ActualWidth]. Is that what you want?

Answer 2

screenX and screenY should be getting computed relative to screen center ...

Answer 3

I don't see a correction for the fact that when drawing using the Windows API, the origin is in the upper left corner of the screen. I am assuming that your coordinate system is

y
|
|
+------x

Also, is your coordinate system assuming origin in the center, per Scott's question, or is it in the lower left corner?

But, the Windows screen API is

+-------x
|
|
|
y

You would need the coordinate transform to go from classic Cartesian to Windows.

Answer 4

Since Windows coordinates are z into the screen (x cross y), I would use something like

screenY = viewPort.ActualHeight * (1 - screenY);

instead of

screenY = screenY * viewPort.ActualHeight;

to correct screenY to accomodate Windows.

Alternately, you could use OpenGL. When you set the viewport x/y/z range, you could leave it in "native" units, and let OpenGL convert to screen coordinates.

Edit: Since your origin is the center. I would try

screenX = viewPort.ActualWidth * (screenX + 1.0) / 2.0
screenY = viewPort.ActualHeight * (1.0 - ((screenY + 1.0) / 2.0))

The screen + 1.0 converts from [-1.0, 1.0] to [0.0, 2.0]. At which point, you divide by 2.0 to get [0.0, 1.0] for the multiply. To account for Windows y being flipped from Cartesian y, you convert from [1.0, 0.0] (upper left to lower left), to [0.0, 1.0] (upper to lower) by subtracting the previous screen from 1.0. Then, you can scale to the ActualHeight.

Answer 5

I've created and succesfully tested a working method by using the 3DUtils Codeplex source library.

The real work is performed in the TryWorldToViewportTransform() method from 3DUtils. This method will not work without it (see the above link).

Very useful information was also found in the article by Eric Sink: Auto-Zoom.

NB. There may be more reliable/efficient approaches, if so please add them as an answer. In the meantime this is good enough for my needs.

    /// <summary>
    /// Takes a 3D point and returns the corresponding 2D point (X,Y) within the viewport.  
    /// Requires the 3DUtils project available at http://www.codeplex.com/Wiki/View.aspx?ProjectName=3DTools
    /// </summary>
    /// <param name="point3D">A point in 3D space</param>
    /// <param name="viewPort">An instance of Viewport3D</param>
    /// <returns>The corresponding 2D point or null if it could not be calculated</returns>
    public Point? Point3DToScreen2D(Point3D point3D, Viewport3D viewPort)
    {
        bool bOK = false;

        // We need a Viewport3DVisual but we only have a Viewport3D.
        Viewport3DVisual vpv =VisualTreeHelper.GetParent(viewPort.Children[0]) as Viewport3DVisual;

        // Get the world to viewport transform matrix
        Matrix3D m = MathUtils.TryWorldToViewportTransform(vpv, out bOK);

        if (bOK)
        {
            // Transform the 3D point to 2D
            Point3D transformedPoint = m.Transform(point3D);

            Point screen2DPoint = new Point(transformedPoint.X, transformedPoint.Y);

            return new Nullable<Point>(screen2DPoint);
        }
        else
        {
            return null; 
        }
    }

Answer 6

This doesn't address the algoritm in question but it may be useful for peple coming across this question (as I did).

In .NET 3.5 you can use Visual3D.TransformToAncestor(Visual ancestor). I use this to draw a wireframe on a canvas over my 3D viewport:

    void CompositionTarget_Rendering(object sender, EventArgs e)
    {
        UpdateWireframe();
    }

    void UpdateWireframe()
    {
        GeometryModel3D model = cube.Content as GeometryModel3D;

        canvas.Children.Clear();

        if (model != null)
        {
            GeneralTransform3DTo2D transform = cube.TransformToAncestor(viewport);
            MeshGeometry3D geometry = model.Geometry as MeshGeometry3D;

            for (int i = 0; i < geometry.TriangleIndices.Count;)
            {
                Polygon p = new Polygon();
                p.Stroke = Brushes.Blue;
                p.StrokeThickness = 0.25;
                p.Points.Add(transform.Transform(geometry.Positions[geometry.TriangleIndices[i++]]));
                p.Points.Add(transform.Transform(geometry.Positions[geometry.TriangleIndices[i++]]));
                p.Points.Add(transform.Transform(geometry.Positions[geometry.TriangleIndices[i++]]));
                canvas.Children.Add(p);
            }
        }
    }

This also takes into account any transforms on the model etc.

See also: http://blogs.msdn.com/wpf3d/archive/2009/05/13/transforming-bounds.aspx

Answer 7

Hi guys, I have a similar problem but this algorithm is working for me as my camera is moving along a curved path (constantly setting camera position and lookat position) so the screen plane changes position/orientation as the camera position and lookat changes.

My original posting is here. Does anyone have a function that takes this into account?

thanks