WebXR Input Sources

[Maksims]

This PR adds WebXR Input Sources, which provides interface to various input methods in VR and AR environments.

It is WIP PR, so comments and ideas are welcome.

New APIs:

app.xr.input // instance of pc.XrInput

// pc.XrInput
app.xr.input.inputSources // list of active input sources
app.xr.input.on('add', function(inputSource) { }); // event when pc.XrInputSource is added
app.xr.input.on('remove', function(inputSource) { }); // event when pc.XrInputSource is removed
app.xr.input.on('select', function(inputSource, evt) { }); // when device is triggered. This could be handheld main trigger button, or screen tap on mobile
app.xr.input.on('selectstart', function(inputSource, evt) { }); // when devices starts to be triggered
app.xr.input.on('selectend', function(inputSource, evt) { }); // when devices ends trigger

// pc.XrInputSource
inputSource.ray // pc.Ray in XR session space, origin and direction of the pointing device
inputSource.grip // True if input source can be held
inputSource.position // pc.Vec3 | null - position of handheld device in XR session space
inputSource.rotation // pc.Quat | null - rotation of handheld device in XR session space
inputSource.selecting // True if input source is in primary action state, such as trigger button is pressed or screen is being touched.
inputSource.gamepad // null or instance of Gamepad. This is available on platforms with support of https://www.w3.org/TR/webxr-gamepads-module-1/
inputSource.targetRayMode // string, one of pc.XRTARGETRAY_*
inputSource.handedness // string, one of pc.XRHAND_*
inputSource.profiles // array of strings, lists of profiles reported by device of its visual and behavioural representation. Refer to official documentation: https://www.w3.org/TR/webxr/#dom-xrinputsource-profiles
inputSource.inputSource // XRInputSource - raw object from WebXR
inputSource.on('remove', function() { }); // when device is removed/disconnected
inputSource.on('select', function(evt) { }); // when device is triggered. This could be handheld main trigger button, or screen tap on mobile
inputSource.on('selectstart', function(evt) { }); // when devices starts to be triggered
inputSource.on('selectend', function(evt) { }); // when device ends trigger

// pc
pc.XRTARGETRAY_GAZE // 'gaze' - indicates the target ray will originate at the viewer and follow the direction it is facing. (This is commonly referred to as a "gaze input" device in the context of head-mounted displays.)
pc.XRTARGETRAY_SCREEN // 'screen' - indicates that the input source was an interaction with the canvas element associated with an inline session’s output context, such as a mouse click or touch event.
pc.XRTARGETRAY_POINTER // 'tracked-pointer' - indicates that the target ray originates from either a handheld device or other hand-tracking mechanism and represents that the user is using their hands or the held device for pointing.

pc.XRHAND_NONE // 'none' - input source is not meant to be held in hands.
pc.XRHAND_LEFT // 'left' - indicates that input source is meant to be held in left hand.
pc.XRHAND_RIGHT // 'right' - indicates that input source is meant to be held in right hand.

I confirm I have signed the Contributor License Agreement.

[Maksims]

Here is basic example, that provides simple entity picking by bounding box, that works with multi-controllers (Oculus Quest for example); gaze (mobile devices); screen (AR taps on screens).

select.js (attach on entity with model):

var Select = pc.createScript('select');

Select.prototype.initialize = function() {

    // when select is triggered
    this.app.xr.input.on('select', function(inputSource) {

        // get mesh of this entity
        var mesh = this.entity.model.meshInstances[0];

        // check if input source ray intersects with mesh bounding box
        if (mesh.aabb.intersectsRay(inputSource.ray)) {

            // randomize its diffuse color
            mesh.material.diffuse.set(Math.random(), Math.random(), Math.random());
            mesh.material.update();
        }
    }, this);
};

[Maksims]

Here is basic example for adding/removing input sources and positioning entity based on grip position and rotation of a physical controller.

controllers.js: (somewhere on root)

var Controllers = pc.createScript('controllers');

// entity for controller template
Controllers.attributes.add('template', {
    type: 'entity'
});

Controllers.prototype.initialize = function() {
    // when input source added
    this.app.xr.input.on('add', function(inputSource) {
        var entity = this.template.clone();
        entity.script.controller.inputSource = inputSource;
        entity.reparent(this.app.scene.root);
        entity.enabled = true;
    }, this);
};

controller.js: (on controller entity)

var Controller = pc.createScript('controller');

Controller.attributes.add('inputSource', { type: 'string' });

Controller.prototype.initialize = function() {
    this.inputSource.on('remove', function() {
        // destroy self when input source is removed
        this.entity.destroy();
    }, this);
};

Controller.prototype.update = function(dt) {
    if (this.inputSource.grip) {
        // if can be gripped
        this.entity.model.enabled = true;
        this.entity.setLocalPosition(this.inputSource.position);
        this.entity.setLocalRotation(this.inputSource.rotation);
    } else {
        // some controllers cannot be gripped
        this.entity.model.enabled = false;
    }
};