Begin migration of dev docs to GitHub

docs/mrtk3-overview/api-reference.md

@@ -0,0 +1,15 @@
+# API reference — MRTK3
+
+MRTK3 is now shipping as [a set of individually versioned packages](index.md#versioning). As a result, each MRTK3 package ships its API reference individually. Please find the links to the API references for MRTK3 packages below:
+
+- [MRTK Core Definitions](https://aka.ms/mrtk3coreapi)
+- [MRTK Accessibility](https://aka.ms/mrtk3accessibilityapi)
+- [MRTK Audio Effects](https://aka.ms/mrtk3audioapi)
+- [MRTK Data Binding and Theming](https://aka.ms/mrtk3dataapi)
+- [MRTK Diagnostics](https://aka.ms/mrtk3diagnosticsapi)
+- [MRTK Graphics Tools](https://learn.microsoft.com/dotnet/api/Microsoft.MixedReality.GraphicsTools)
+- [MRTK Input](https://aka.ms/mrtk3inputapi)
+- [MRTK Spatial Manipulation](https://aka.ms/mrtk3spmanipapi)
+- [MRTK UX Components](https://aka.ms/mrtk3uxcompapi)
+- [MRTK UX Core](https://aka.ms/mrtk3uxcoreapi)
+- [MRTK Windows Speech](https://aka.ms/mrtk3winspeechapi)

docs/mrtk3-overview/architecture/architecture.md

@@ -0,0 +1,40 @@
+# Architecture overview — MRTK3
+
+![Architecture MRTK3](../images/MRTK_v3_Architecture.png)
+
+One of the goals with MRTK3 was to take everything we've learned from the start of MRTK2 back in early 2018, combine it with the work that's been done by our industry partners across OpenXR and Unity since then, and come out the other side with a strong, extensible foundation that allows MRTK to focus more on providing differentiators and an overall improved user (and developer!) experience.
+
+## Input and interactions
+
+The overall architecture of the input stack of MRTK3 is built on four foundational components:
+
+1. OpenXR
+   1. [Khronos Specification](https://www.khronos.org/registry/OpenXR/specs/1.0/html/xrspec.html)
+   1. [Unity OpenXR Plugin documentation](https://docs.unity3d.com/Packages/com.unity.xr.openxr@latest)
+1. [Unity subsystems](https://docs.unity3d.com/ScriptReference/UnityEngine.SubsystemsModule.html)
+1. [Unity's Input System](https://docs.unity3d.com/Packages/com.unity.inputsystem@latest)
+1. [Unity's XR Interaction Toolkit](https://docs.unity3d.com/Packages/com.unity.xr.interaction.toolkit@latest)
+
+along with a layer of MRTK-defined [interactors](interactors.md) and [subsystems](subsystems.md), providing features like poke and speech.
+
+### OpenXR
+
+OpenXR is the interface between an application and an XR runtime system, allowing for a common set of features to be called generically and allow the hardware-specific implementation to be handled by the XR runtime. Adopting this in MRTK3, along with Unity's Input System, lets Unity handle more of the cross-platform and extensible input story while allowing MRTK3 to focus on helping you build rich experiences on top.
+
+### Subsystems
+
+[Subsystems](subsystems.md) and Unity's [SubsystemManager](https://docs.unity3d.com/ScriptReference/SubsystemManager.html) should be conceptually familiar to MRTK2 users, as they're the new "data providers". The idea is that different platforms or services can provide an implementation of a specific type of MRTK subsystem and have that run when relevant, providing data to MRTK3 and the app overall, just like data providers did to the various systems in MRTK2. Since we're focusing on OpenXR, the goal is that many features are covered by a cross-vendor EXT extension in OpenXR and multiple subsystems aren't needed, but vendor-specific extensions can represent early tech advancements that we want to support.
+
+### Unity Input System
+
+Conceptually, Unity's Input System will also feel familiar to users of the MRTK2 controller mapping profile. It provides a central place for mapping the buttons and other input axes on a controller or hand to a set of actions. These actions are then consumed by Unity's XR Interaction Toolkit (XRI) and MRTK3, so the object being interacted with doesn't care as much about _what_ is manipulating it, just that it _is_.
+
+### XR Interaction Toolkit
+
+XRI provides a foundation of interactors and interactables. MRTK builds upon this with its own set of [interactors](interactors.md) and [interactables](interactables.md), allowing for additional features like articulated hand tracking, gaze, and pinch.
+
+## See also
+
+- [Subsystems](subsystems.md)
+- [Interactors](interactors.md)
+- [Interactables](interactables.md)

docs/mrtk3-overview/architecture/interactables.md

@@ -0,0 +1,55 @@
+# Interactables — MRTK3
+
+MRTK builds on the `XRBaseInteractable` provided by Unity's XR Interaction Toolkit. The existing interactable behavior and API is fully supported in MRTK, and all of our custom interactables obey the existing XRI interactable API.
+
+For developers new to XRI, we _strongly_ recommend that you first review Unity's [XRI architecture documentation](https://docs.unity3d.com/Packages/com.unity.xr.interaction.toolkit@2.0/manual/architecture.html).
+
+To expand upon the interactable mechanisms included in XRI, MRTK offers two base classes upon which advanced interactions can be built, one extending the other.
+
+![Interactables inheritance diagram](images/interactable_classes.svg)
+
+- `MRTKBaseInteractable : XRBaseInteractable`
+  - This class offers filtering and flagging for different types of interactors. While the base XRI `XRBaseInteractable` doesn't discriminate between interactor types, `MRTKBaseInteractable` provides convenience functions for checking whether common types of interactions are occurring. Convenience properties like `IsGazeHovered` or `IsGrabSelected` are shortcuts to querying whether a participating interactor implements a given interface (correspondingly, `IGazeInteractor` or `IGrabInteractor`). These flags are more performant than iterating through the list of `interactorsHovering` or `interactorsSelecting`. In addition, `MRTKBaseInteractable` can filter/reject certain types of interactors in the case that the developer wishes to exclude certain input modalities.
+- `StatefulInteractable : MRTKBaseInteractable`
+  - While `MRTKBaseInteractable` adds flags and filters, and avoids adding any additional state to the interactable, `StatefulInteractable` introduces useful stateful features like toggling and variable selection.
+
+## Strict separation of state and visuals
+
+In MRTK 2.x, interactables were often responsible for driving their own visual effects, be it the compressing of a 3D button, a hover effect, or even just changing color on a click. The limitation of this approach is that the interaction logic is tightly bound to the visuals. If you were to redesign the visuals or use a different size/shape/displacement/etc. of button, the interaction script itself would need to change.
+
+In MRTK3, **interactables are pure state and interaction.** The interactable doesn't render any visual changes or effects based on its internal state. It's purely a collection of state and interaction logic that's highly portable between visual presentation setups.
+
+![Strict isolation of state and visuals](images/pressable.png)
+
+The same `PressableButton` script can be used to build a squishy ball, a pressable "trackpad"-like plane, or an abstract pressable that issues network events on press. The `PressableButton` script doesn't even care "where" it is; it could be inside a Canvas, or on a rigidbody.
+
+To drive visuals, a separate "visual driver" is used to poll the state from the interactable and render the appropriate feedback. `StateVisualizer` is the recommended low-code method for driving common visual feedback effects from interactable state, but developers are free to write their own custom visual drivers. For example, our button components generally use `StateVisualizer` for their advanced 3D + shader-based feedback effects, but we also provide an example `BasicPressableButtonVisuals` that shows how a simple visual driver can be authored in code.
+
+## Variable selection
+
+`StatefulInteractable`'s most useful additional feature over the base XRI functionality is support for variable `Selectedness`. While base XRI interactables are either selected or not selected, MRTK's `StatefulInteractable`s can be any floating-point fraction of selected.
+
+This concept is useful when working in XR, since nearly all forms of input are no longer binary states. Motion controllers often have analog triggers (or analog grips!), hand interactions can provide a variable "pinchedness", and volumetric press interactions can depress a button or pushable surface by a varying amount. You see these variable, analog interactions everywhere in XR, and MRTK is equipped to help developers build delightful interactions on top of these analog inputs.
+
+A wide range of different interactors and types of interactions can all contribute together to the overall Selectedness of an interactable. Notably, all interactors that implement `IVariableSelectInteractor` contribute their analog selection amount, typically through a `max()` of all participating interactors. This variable amount is combined with the binary, non-variable selections coming from vanilla-style interactors.
+
+For derived classes like `PressableButton`, the `Selectedness()` function is overridden to add an additional "ingredient" to the selectedness computation. Interactors that implement `IPokeInteractor` can contribute Selectedness based on their physical location and how they're physically pressing down on the interactable. Other derived classes can introduce other, arbitrary forms of selection.
+
+![Variable selectedness](images/selectedness.svg)
+
+For the interactables MRTK provides, `Selectedness()` and `isSelected` will always "agree"--in other words, you'll never observe a `Selectedness()` greater than the `SelectThreshold` without a corresponding XRI `isSelected` and an accompanying interactor in `interactorsSelecting`.
+
+> [!IMPORTANT]
+> Your custom interactable subclasses can obviously override `Selectedness` to some other value that's completely disconnected from the XRI `isSelected`. However, our interactables don't do this, and we strongly discourage it. **In general, never write _interactions_ that do not have a corresponding _interactor_.** XRI selection will, in the vast majority of cases, be sufficient, and any custom interactions you build should be written as interactors.
+
+When you're creating a custom interactable that supports a new method of determining `Selectedness()`, simply override the method and combine your new selectedness with the existing selection amount. If you're using `StateVisualizer` or any other visual layer that listens to variable selection, it will respond accordingly to your new selection type.
+
+## Map UGUI events to XRI
+
+In some cases, it's desirable to have interactables respond to UGUI events, such as mouse, gamepad, or touchscreen input. The `UGUIInputAdapter`, which is a UGUI `Selectable`, receives UGUI events and forwards them to a `CanvasProxyInteractor`, if one is present.
+
+![UGUI adapter flow](images/UGUI.svg)
+
+When the `CanvasProxyInteractor` is notified of the UGUI events by the `UGUIInputAdapter`, it issues _equivalent_ XRI actions on the relevant interactable. The mapping between UGUI input and XRI actions is somewhat lossy and is an area of active development.
+
+With this system, existing XRI interactables that are built for immersive platforms, hands, motion controllers, and 3D input can react equally well to accessible 2D controls like mouse and gamepad.