Migrating to MRKT2 - Interaction with irregular or complex objects

Intro

Interaction with objects that are not simple shapes like cubes, spheres, capsules etc. poses some challenges. The Mixed Reality Toolkit 2 offers some great components, but they all require a top level collider. Now consider this helicopter:

It consists of a lot of small objects. Default it does not even have a collider. You cannot add a Near Interaction Touchable on top of the object because it simply cannot find a collider. Now you can generate those on import, but that makes the object kind of heavy with regards to required processing power, and hooking all those colliders up up to their own Interaction Touchable is a lot of work.

There is a simpler way of doing that, fortunately. Actually, there are two variants of this, but I am going to show the one I think works the best (and is the most beautiful).

Adding a colliding 'catcher' to rule them all

First of all, we are going to add a surrounding object inside the model itself. I took a capsule, as this gives IMHO the most beautiful result

The result now is the helicopter is now almost completely covered in what looks like a giant suppository, which is definitely not what you want.

So by fiddling around I created this material. Note: it actual color is fully transparent black - basically 0,0,0,0. so that we can see the helicopter again.

But more importantly, it has a hover light override color of green with a light intensity of 0.4.

And now, if a cursor strikes the object, you get this what I think is rather pretty ghostly glow indicating this is your focused object.

Making it interactable

Making it actually interact with events is now pretty simple. Assuming your actual 'controller' behaviour needs to control the whole game object, it needs to sit on top - so it can do more things than just interaction (like moving the helicopter, for instance - which it does not do now). It looks like this:

using System;
using TMPro;
using UnityEngine;

public class InteractionResponder : MonoBehaviour
{
    [SerializeField]
    private TextMeshPro _text;

    private int _timesClicked;

    private int _timesTouched;

    private int _timesFocus;

    public void Click()
    {
        _timesClicked++;
        UpdateText();
    }
    
    public void TouchStart()
    {
        _timesTouched++;
        UpdateText();
    }

    public void OnFocus()
    {
        _timesFocus++;
        UpdateText();
    }

    private void UpdateText()
    {
        _text.text = string.Format("Clicked: {0}{1}Touched: {2}{1}Focused: {3}", 
            _timesClicked, Environment.NewLine, _timesTouched, _timesFocus);
    }
}

Super simple, basically only three event response methods that can be called - and it tries to display the text in a TextMeshPro object, that also sits in the HologramCollection, just like the Helicopter it self. As I said, the InteractionResponder behaviour will be sitting on the helicopter object itself:

Now we need go back to the SurroundingCapsule and add an Interactable and a Near Interaction Touchable Volume script to that. The latter is apparently new or something I missed: where an ordinary Near Interaction Touchable only takes a rectangular collider, the Volume script also takes a capsule:

Then, you will need to select "Select" for Input Actions, then drag the helicopter in the little box under OnClick like you usually do in this kind of event hookup, and select ÏnteractionResponder.Click.

Then you click "Add Event", select "InteractableOnTouchReceiver" and hook the On Touch event up to the InteractionResponder.TouchStart method (we will ignore the On Touch End event in this sample).

In a similar fashion, you will add another event, select "InteractableOnFocusReceiver" and hook that to the InteractionResponder.OnFocus event.

And if you have done everything right, and hit the play button in the editor, you will see this happening when you click, touch or focus:

Conclusion

With very little code and one extra component you can make an irregular and complex object like a helicopter have an easy to touch/focus/click target that also gives some subtle but very clear visual feedback about what is happening. And this is only a start - we might as well have it respond to touch by showing some direct feedback, or show that it knows it's focused, even without a cursor hitting it (eye gaze!). Life is going to get interesting and much more immersive once HoloLens 2 comes around!

Demo project can be found here.