Calculate Mixed Reality object locations on top or in front of obstructions using BoxCastAll

Intro

It was quite difficult to name this article. Usually I try to find a title that more or less describes a good search term that I used myself when I was looking for ‘something like this’, but I could not really find what I was looking for. What I have here is code that calculates locations for objects to be in front or on top of other objects and/or the Spatial Mesh. It does so using a BoxCastAll, something I have tried to use before, but not very successfully.  I have tried using Rigidbody.SweepTest and although it works for some scenarios, it did not work for all. My floating info screens ended up half a mountain anyway (in Walk the World), or the airport could not move over the ground because of some tiny obstacle blocking it (in AMS HoloATC). So I tried a new approach.

This is part one of a two-post blog post. Explaining how the BoxCast works and what extra tricks and calculations were necessary to get it to work properly proved to need quite a long text, so I will leave the behaviour actually using this code in good way for the next post.

BoxCast magic

So what is a BoxCast, actually? It’s comparable to a normal RayCast. But where a RayCast gives you the intersection of a line and an obstruction, a BoxCast does that – suprise - with a box. You essentially throw a box from a point along a vector until it hits something – or some things, as a BoxCastAll potentially returns more than one hit. If you take the one that is closest to your camera (a hit has a convenient “distance” property) you potentially have a place where you can place the object.

… except that it does not take into account the following things:

• An object’s (center) position and the center of it’s bounding box are not always the same; this will make the BoxCast not always happen at the place you think it does
• The vector from the camera to the hit may or may not be parallel to the direction of the BoxCast; therefore, we need to project the vector from the camera to the hit on the vector of the BoxCast.
• The BoxCast hit detection happens at the edge of the casted box, and an objects position is determined by it’s center – so, we need so we need to move back a little towards the camera, or else about half of our object – determined by it’s actual orientation - will end up inside the obstruction.

My code takes all of that into account. It was quite hard won knowledge before I uncovered all the lovely pitfalls.

First a new utility method

For a BoxCast to work, you need a box. You typically accomplish that by getting the bounds of all the Renderers in the object you want to cast and combine those to one big bounding box. I hate typing or copying code more than once, so I create this little extension method to GameObject

public static class GameObjectExtensions
{
    public static Bounds GetEncapsulatingBounds(this GameObject obj)
    {
        Bounds totalBounds = new Bounds();

        foreach (var renderer in obj.GetComponentsInChildren<Renderer>())
        {
            if (totalBounds.size.magnitude == 0f)
            {
                totalBounds = renderer.bounds;
            }
            else
            {
                totalBounds.Encapsulate(renderer.bounds);
            }
        }

        return totalBounds;
    }
}

BoxCast Magic

In LookingDirectionHelpers, a static class containing utilities to calculate directions and places the direction the user is looking (duh) at I have created a method that does the BoxCast magic. It does quite a lot, and I am going through it step by step. It starts like this:

public static Vector3 GetObjectBeforeObstruction(GameObject obj, float maxDistance = 2,
    float distanceFromObstruction = 0.02f, int layerMask = Physics.DefaultRaycastLayers,
    BaseRayStabilizer stabilizer = null, bool showDebugLines = false)
{
    var totalBounds = obj.GetEncapsulatingBounds();

    var headRay = stabilizer != null
        ? stabilizer.StableRay
        : new Ray(CameraCache.Main.transform.position, CameraCache.Main.transform.forward);

     var hits = Physics.BoxCastAll(GetCameraPosition(stabilizer),
                                  totalBounds.extents, headRay.direction,
                                  Quaternion.identity, maxDistance, layerMask)
                                  .Where(h => !h.transform.IsChildOf(obj.transform)).ToList();

As you can see, the method accepts quite some parameters, most of them optional:

• obj – the actual object to cast and place against or on top of the obstruction
• maxDistance – the maximum distance to place the object from the camera (if it does not hit another object first)
• distanceFromObstruction – the distance to keep between the object and the obstruction
• layerMask – what layers should we ‘hit’ when we are looking for obstructions (default is everything)
• stabilizer – used to get a more stable location and viewpoint source than the camera itself
• showDebugLines – use some awesome help classes I nicked from the Unity Forums from “HiddenMonk” to show how the BoxCast is performed. Without these, I sure as hell would not have been able to identify all issues that I had to address.

Well then – first we get the total encapsulating bounds, then we check if we can either use the Stabilizer that we need to use the camera to define a ray in the direction we want to cast. The we calculate a point dead ahead of the camera – this is the.

And then we do the actual BoxCast, or actually a BoxCastAll. The cast is done:

• From the Camera position
• Using the total extents of the object
• In the direction of the viewing ray (so a line from your head to where the gaze cursor is)
• using no rotation (we used the Render's bounds, that already takes any rotation into account)
• over a maximum distance
• against the layers described by the layer mask (default is all)

Notice the Where clause at the end. BoxCasts hit everything, including child objects of the cast object itself, as it may be in the path of it’s own cast. So we need to weed out any hits that apply to the object itself or its children.

The next piece of visualizes how the BoxCast is performed, using HiddenMonk's code:

if (showDebugLines)
{
    BoxCastHelper.DrawBoxCastBox(GetCameraPosition(stabilizer),
        totalBounds.extents, headRay.direction,
        Quaternion.identity, maxDistance, Color.green);
}

This uses Debug.Draw – these lines are only visible in the Unity editor, in Play mode. They will not show up in the Game pane but in the Scene pane. Which makes sense, as you can them look at the result from every angle without affecting the actual scene in the game.

This looks like this:

Now to address the issues I listed on top of this article, we need to do a few things.

Giving it the best shot cast

The next line is a weird one but is explained by the fact that there may be a difference between the center of the actual bounding box (and thus the cast) and center of the object as reported by Unity. I am not entirely sure why this is, but trust me, it's happens with some objects. We need to compensate for that.

var centerCorrection = obj.transform.position - totalBounds.center;

Below you see an example of such an object. I typically happens when an object is composed of one or more other objects that are off center, and especially when the object is asymmetrical. Like this 'floating screen'. You will see it's an empty game object containing a Quad and a 3DTextPrefab that are moved upwards in local space. Without the correction factor, you get the situation on the left - the BoxCast happens 'too low'

On the right side, you see the the desired effect. I opted to change the location to of the object to the center of the BoxCast – you might also consider changing the start location of the BoxCast, but that a side effect: the ray won’t start at the user’s viewpoint (but in this case, a little bit above it) which might be confusing or produce undesirable results.

Hit or miss - projection

We need to find the closest hit… but that hit might not be right in front us, along the viewing vector. So  we need to create a vector from the camera to the hit, then make a (longer) vector that follows the user’s gaze, and finally project the ‘hit vector’ to the ‘gaze vector’. Then and only then we know how much room there is in front of us.

if (hits.Any())
{
    var closestHit = hits.First(p => p.distance == hits.Select(q => q.distance).Min());
    var hitVector = closestHit.point - GetCameraPosition(stabilizer);
    var gazeVector = CalculatePositionDeadAhead(closestHit.distance * 2) - 
                       GetCameraPosition(stabilizer);
    var projectedHitVector = Vector3.Project(hitVector, gazeVector);

To show what happens, I have made a screenshot where I made Unity draw debug lines for every calculated vector:

if (showDebugLines)
{
    Debug.DrawLine(GetCameraPosition(stabilizer), closestHit.point, Color.yellow);
    Debug.DrawRay(GetCameraPosition(stabilizer), gazeVector, Color.blue);
    Debug.DrawRay(GetCameraPosition(stabilizer), projectedHitVector, Color.magenta);
}

Which results in the following view (for clarity I have disabled the code that draws the BoxCast for this screenshot)

A little magnification shows the area of interest a little bit better:

You can clearly see the the yellow line from the camera to the original hit, the blue line which is the viewing direction of the user, and the magenta line projected on that.

Keep your distance please

Now this all works fine for a flat object like a Quad (posing as an 'info screen' here. But not on a box like this for instance (which I made partially translucent for clarity).

The issue here is simple, although it took me some time to figure out what was causing it. Like I said before, the hit takes place at the edge of the shape, but the object's position is tied to it's center, so if I set the object's position to that hit, it will end up halfway the obstruction. QED.

So what we need to do is make yet another ray, that will go from the center of the object to the edge, following the same direction as the projected hit vector (the magenta line). Now RayCasts don't work from inside an object, but fortunately there's another way - the Bounds class supports an IntersectRay method. It works a bit kludgy IMHO but it does the trick:

var edgeRay = new Ray(totalBounds.center, projectedHitVector);
float edgeDistance;
if(totalBounds.IntersectRay(edgeRay,  out edgeDistance))
{
    if (showDebugLines)
    {
        Debug.DrawRay(totalBounds.center, 
            projectedHitVector.normalized * Mathf.Abs(edgeDistance + distanceFromObstruction),
            Color.cyan);
    }
}

So we intersect the projected hit vector from the center of the bounds to the edge of the bounds. This will give us the distance from the center to the part of the object that hit the obstruction, and we can move the object 'backwards' to the desired position. Since I specified a 'distanceFromObstruction' we can add that to the distance the object needs to be moved 'back'  as well to keep a distance from an obstruction, in stead of touching it (although for this object it's 0). Yet another debug line, cyan this time, shows what's happening:

The cyan line is the part over which the object is moved back. Now the only thing left is to calculate the new position and return it, this time using the centerCorrection we used before to make the object actually appear within the BoxCast's 'outlines':

return GetCameraPosition(stabilizer) +
            projectedHitVector - projectedHitVector.normalized * 
            Mathf.Abs(edgeDistance + distanceFromObstruction) +
            centerCorrection;

Nobody is perfect

If you think "hey, it looks like it is not completely perfectly aligned", you are right. This is because Unity has it's limits in determining volumes and bounding boxes. This is probably because the main concern of a game is performance, not 100% accuracy. If I add this line to the code

BoxCastHelper.DrawBox(totalBounds.center, totalBounds.extents, Quaternion.identity, Color.red);

it actually shows the bounding box:

So this explains a bit more what is going on. With all the debug lines enabled it looks like this, which I can imagine is as confusing as helpful ;)

Show and tell

It’s actually not really easy to properly show you how this method can be utilized. As I said in the beginning, I will save that for the next post. In the mean time, I have cobbled together a demo project that uses the GetObjectBeforeObstruction in a very simple way. I have created a SimpleKeepInViewController that polls every so many seconds (2 is default) where the user looks, then calls GetObjectBeforeObstruction and moves the object there. This gives a bit of a nervous result, but you get the idea.

public class SimpleKeepInViewController : MonoBehaviour
{
    [Tooltip("Max distance to display object before user")]
    public float MaxDistance = 2f;

    [Tooltip("Distance before the obstruction to keep the current object")]
    public float DistanceBeforeObstruction = 0.02f;

    [Tooltip("Layers to 'see' when detecting obstructions")]
    public int LayerMask = Physics.DefaultRaycastLayers;

    [Tooltip("Time before calculating a new position")]
    public float PollInterval = 2f;

    [SerializeField]
    private BaseRayStabilizer _stabilizer;

    [SerializeField]
    private bool _showDebugBoxcastLines = true;

    private float _lastPollTime;


    void Update()
    {
        if (Time.time > _lastPollTime)
        {
            _lastPollTime = Time.time + PollInterval;
            LeanTween.move(gameObject, GetNewPosition(), 0.5f).setEaseInOutSine();
        }
#if UNITY_EDITOR
        if (_showDebugBoxcastLines)
        {
            LookingDirectionHelpers.GetObjectBeforeObstruction(gameObject, MaxDistance,
                DistanceBeforeObstruction, LayerMask, _stabilizer, true);
        }
#endif
    }

    private Vector3 GetNewPosition()
    {
        return LookingDirectionHelpers.GetObjectBeforeObstruction(gameObject, MaxDistance,
            DistanceBeforeObstruction, LayerMask, _stabilizer);
    }
}

There is only one oddity here – you see I actually call GetObjectBeforeObstruction twice. But the first time only happens in the editor, and only if you select the Show Debug Boxcast Line checkbox:

If I did not add this, you would see the lines flash for one frame every 2 seconds, which is hardly enlightening. This way, you can see them all the time in the editor

In the demo project you will find three objects – in the images above you have already seen a single block (the default), a rotating ‘info screen’ that shows “Hello World” and there’s also this composite object on the left (two cubes off-center), here displayed with all debug lines enabled ;). You can toggle between the three objects by saying “Toggle” or by pressing the “T”. The latter will actually also work in a HoloLens if you have a Bluetooth keyboard attached - and believe me, I tried ;-)

Conclusion

Yet another way to make an object appear next to on on top of an obstruction ;). This code actually took me way too much time to complete, but I learned a lot from it and at some point it became a matter of honor to get the bloody thing to work.

Fun factoid: most of the code, and a big part of the blog post, was actually written on train trips to an from an awesome HoloLens project I am currently involved in. Both the demo project and this blog post were actually published while being on my way, courtesy of the Dutch Railways free WiFi service ;)