Many times, you will need to know the 2D screen coordinates of a 3D world position. DirectX already includes methods to perform vector projections, taking into account the needed World, View and Projection matrices, as well as the viewport scaling. It does not include however viewport clipping, as an additional feature in those methods.
Viewport clipping can be a tricky matter, and sometimes, you will need to rely on algorithms like the Sutherland-Hodgman algorithm, or the refined version specifically developed for 2D viewports: the Cohen-Sutherland algorithm. Those methods are especially appropriate when you are already dealing with 2D coordinates, or if you need to know the extra points or polygons generated when clipping is performed.
In our case however, we will only focus on finding the closest in-screen coordinates that correspond to an off-screen point, without dealing with any extra geometry or polygon sub-division. It’s important to note also that we will be working with 3D coordinates that go through a projection process (and finally getting 2D coords). This is relevant, as provides us with additional information we can use, and allows us to jump inside the algorithm and perform the clipping in the middle of the projection pipeline, instead of doing so at the end, when the coordinates are already 2D.
As you can see, each 3D position travels through different stages and spaces of coordinates: model space –> world space -> camera space –> projection space –> clipping space –> homogeneous space –> and finally: Screen Space.
Evidently, D3D also performs certain types of clipping to vectors, and you can tell by the above picture that clipping is done, (surprisingly), in clip space. We will try to mimic that behavior…
Note: Transforming coordinates with the MClip matrix, to go from projection space to clip space should be done only if you want to scale or shift your clipping volume. If you are ok with a clipping volume that matches your screen render target viewport (you will, most of the cases), you should leave this matrix as the Identity, or simply don´t perform this step. The below written algorithm has all this step commented.
Once our coordinates are in Clip Space (Xp, Yp, Zp, Wp), we easily perform the clipping by limiting their values to the range: –Wp .. Wp for the X and Y, and to the range: 0 .. Wp for Z.
After that, we just need to proceed with the normal Vector projection algorithm, as the resulting 2D coordinates will be stuck inside the screen viewport. An extra feature that should be nice to have, is a simple output variable that tells us if the coordinates were inside or outside the viewport.
A C# implementation of such an algorithm could be:
Hope it helps !