As AR-capable WebXR implementations come online, it will hopefully be possible to do custom computer vision in the browser. Web assembly is plenty fast; the limitation is efficient access to the image/depth data + camera intrinsics + camera extrinsics from any cameras on the device, expressed in a form that is compatible with the WebXR device data. Easy synchronization with other possible sensors (e.g., accelerometers, etc).

It seems there are two ways to approach this:

• expose the data each device frame, synchronously, through functionality in WebXR (either the Device API or a new associated API)
• expose the data asynchronously, using different api's (e.g., extensions to Sensor and WebRTC APIs?), but in a way that supports synchronization

As AR-capable devices (existing ones exposing APIs like Windows MR, ARKit, etc., or new devices with new APIs) become more pervasive, it should be reasonable to assume that an AR-capable device has the necessary data (camera and other sensor specs in relation to the display, an API that makes efficient access possible) to power such capabilities.

While it will not be necessary for all web-based AR applications to do custom CV, a wide variety of applications will need to. This is especially true if WebXR does not provide a standard set of capabilities that all platforms must implement.