Tuesday, 29 January 2013

Projecting a 3D Vector to 2D screen space, with automatic viewport clipping (DirectX, SlimDX or XNA)

Many times, you will need to know the 2D screen coordinates of a 3D world position. DirectX already includes methods to perform vector projections, taking into account the needed World, View and Projection matrices, as well as the viewport scaling. It does not include however viewport clipping, as an additional feature in those methods.

Viewport clipping can be a tricky matter, and sometimes, you will need to rely on algorithms like the Sutherland-Hodgman algorithm, or the refined version specifically developed for 2D viewports: the Cohen-Sutherland algorithm. Those methods are especially appropriate when you are already dealing with 2D coordinates, or if you need to know the extra points or polygons generated when clipping is performed.

In our case however, we will only focus on finding the closest in-screen coordinates that correspond to an off-screen point, without dealing with any extra geometry or polygon sub-division. It’s important to note also that we will be working with 3D coordinates that go through a projection process (and finally getting 2D coords). This is relevant, as provides us with additional information we can use, and allows us to jump inside the algorithm and perform the clipping in the middle of the projection pipeline, instead of doing so at the end, when the coordinates are already 2D.

Resources like this, and this explain very well the processing of vertices in the Direct3D pipeline:


As you can see, each 3D position travels through different stages and spaces of coordinates: model space –> world space -> camera space –> projection space –> clipping space –> homogeneous space –> and finally: Screen Space.

Evidently, D3D also performs certain types of clipping to vectors, and you can tell by the above picture that clipping is done, (surprisingly), in clip space. We will try to mimic that behavior…

Note: Transforming coordinates with the MClip matrix, to go from projection space to clip space should be done only if you want to scale or shift your clipping volume. If you are ok with a clipping volume that matches your screen render target viewport (you will, most of the cases), you should leave this matrix as the Identity, or simply don´t perform this step. The below written algorithm has all this step commented.

Once our coordinates are in Clip Space (Xp, Yp, Zp, Wp), we easily perform the clipping by limiting their values to the range: –Wp .. Wp for the X and Y, and to the range: 0 .. Wp for Z.

After that, we just need to proceed with the normal Vector projection algorithm, as the resulting 2D coordinates will be stuck inside the screen viewport. An extra feature that should be nice to have, is a simple output variable that tells us if the coordinates were inside or outside the viewport.

A C# implementation of such an algorithm could be:

public static Vector2 ProjectAndClipToViewport(Vector3 pVector, float pX, float pY,
                                float pWidth, float pHeight, float pMinZ, float pMaxZ,
                                Matrix pWorldViewProjection, out bool pWasInsideScreen)
            // First, multiply by worldViewProj, to get the coordinates in projection space
            Vector4 vProjected = Vector4.Zero;
            Vector4.Transform(ref pVector, ref pWorldViewProjection, out vProjected);

            // Secondly (OPTIONAL STEP), multiply by the clipMatrix, if you want to scale
            // or shift the clip volume. If not (most of the times you won´t), just leave 
            // this part commented,

            // or set an Identity Matrix as the clip matrix. The default clip volume parameters
            // (see below), will produce an identity clip matrix.

            //float clipWidth = 2;
            //float clipHeight = 2;
            //float clipX = -1;
            //float clipY = 1;
            //float clipMinZ = 0;
            //float clipMaxZ = 1;
            //Matrix mclip = new Matrix();
            //mclip.M11 = 2f / clipWidth;
            //mclip.M12 = 0f;
            //mclip.M13 = 0f;
            //mclip.M14 = 0f;
            //mclip.M21 = 0f;
            //mclip.M22 = 2f / clipHeight;
            //mclip.M23 = 0f;
            //mclip.M24 = 0f;
            //mclip.M31 = 0f;
            //mclip.M32 = 0;
            //mclip.M33 = 1f / (clipMaxZ - clipMinZ);
            //mclip.M34 = 0f;
            //mclip.M41 = -1 -2 * (clipX / clipWidth);
            //mclip.M42 = 1 - 2 * (clipY / clipHeight);
            //mclip.M43 = -clipMinZ / (clipMaxZ - clipMinZ);
            //mclip.M44 = 1f;
            //vProjected = Vector4.Transform(vProjected, mclip);
            // Third: Once we have coordinates in clip space, perform the clipping,
            // to leave the coordinates inside the screen. The clip volume is defined by:

            //  -Wp < Xp <= Wp
            //  -Wp < Yp <= Wp
            //  0 < Zp <= Wp
            // If any clipping is needed, then the point was out of the screen.
            pWasInsideScreen = true;
            if (vProjected.X < -vProjected.W)
                vProjected.X = -vProjected.W;
                pWasInsideScreen = false;
            if (vProjected.X > vProjected.W)
                vProjected.X = vProjected.W;
                pWasInsideScreen = false;
            if (vProjected.Y < -vProjected.W)
                vProjected.Y = -vProjected.W;
                pWasInsideScreen = false;
            if (vProjected.Y > vProjected.W)
                vProjected.Y = vProjected.W;
                pWasInsideScreen = false;
            if (vProjected.Z < 0)
                vProjected.Z = 0;
                pWasInsideScreen = false;
            if (vProjected.Z > vProjected.W)
                vProjected.Z = vProjected.W;
                pWasInsideScreen = false;

            // Fourth step: Divide by w, to move from homogeneous coordinates to 3D
            // coordinates again

            vProjected.X = vProjected.X / vProjected.W;
            vProjected.Y = vProjected.Y / vProjected.W;
            vProjected.Z = vProjected.Z / vProjected.W;

            // Last step: Perform the viewport scaling, to get the appropiate coordinates
            // inside the viewport

            vProjected.X = ((float)(((vProjected.X + 1.0) * 0.5) * pWidth)) + pX;
            vProjected.Y = ((float)(((1.0 - vProjected.Y) * 0.5) * pHeight)) + pY;
            vProjected.Z = (vProjected.Z * (pMaxZ - pMinZ)) + pMinZ;

            // Return pixel coordinates as 2D (change this to 3D if you need Z)
            return new Vector2(vProjected.X, vProjected.Y);

Hope it helps !



Jobamei said...

Thanks to your explanation , I got a good result !
Here is another question , Could I
back project 2D point at screen coordinate system with z value to world coordinate when lost a w value?
EX: if I have point [u v z ] can I find this point in world coordinate [x y z w]?
Thank a lot

Jobamei said...

Just make w value equal to 1 can solve my problem THX!