As part of coMuse I hope to move towards a fully open-source stack for multi-user musical AR experience. In my last post on this topic, I went through how I set up Linux with Monado and OpenXR to recognise the North Star headset, and utilise its hand and head tracking sensors. This post will be about Stardust XR.
Stardust is a system UI framework where multiple different clients (programs that connect to Stardust directly) can create objects for the user to interact with their apps and device. The user can choose to use the objects that work best for their current scenario, and default setups can easily be created so users start out with all the essentials.
The value of this for coMuse lies in the ability to render and interact with software that exists in 2D forms without having to rebuild or port them to AR. This sits within the developer workflow nicely as it allows for a fully AR setup without switching between AR and 2D displays. Additionally it benefits the participant or audience, as they will likely be experiencing pre-composed Godot scenes or applications built with Stereokit that can sit within Stardust XR.
Some preliminary ideas for developer workflow bindings
- Gestural interactions with already visual languages like PureData
- Adding gestural interactivity for code-based computer music languages, like Supercollider and Glicol
Some preliminary ideas for audience interface with Stardust
- Ease of switching between experiences via panel shells
- Menu style launcher
- Modularity between instruments, which could be their own OpenXR applications.
Revised Linux Stack
North Star → OpenXR → Monado → Stardust XR → Application(s), both 2D and 3D
Installation of Stardust XR
- Install prerequisites found here, i.e.:
yay -S cargo flatbuffers seatd fontconfig wayland openxr-git libglvnd mesa libx11 libxext libxfixes clang --needed
- Download telescope, a bash setup guide for Stardust XR.
git clone https://github.com/StardustXR/telescope
- Enter the folder and run the setup to download Stardust XR’s first party clients.
cd telescope && \ ./setup.sh
- For each first-party client you want to run, enter its folder now downloaded into
repos/and build. We’ll build
cd server && \ cargo build --release && \ cd ../comet && \ cargo build --release
- For ease, lets add the binaries that have been build in each
target/release/folder to our
PATHso that we can run them from anywhere. Add the following to your
~/.zshrcusing a text editor like
nano, taking care to replace
<telescope location>with the absolute path to
export PATH=$PATH$(find <telescope location>/repos/*/target/release/ -maxdepth 1 -type f -executable -printf ":%h")
- Close and open a new terminal window or run
exec zshto refresh. Check that
PATHwas updated with
echo $PATH. You should see the Stardust release folders added at the end.
comet, a first-party Stardust XR client
Stardust XR is an OpenXR client itself, but it runs on top of Monado for now. This means that you must have
monado-service running, so that the server can connect to it, and serve clients.
- Plug in your North Star headset via USB 2.0
- Plug in your North Star via DisplayPort to your graphics card.
portReset()on your North Star, via holding down the circle and dot buttons for 5 seconds.
- Open three terminal windows
monado-servicein the first terminal window.
- A grey, fullscreen Xwayland Monado window should open on your main display. Move it to your North Star display, e.g by using the Kwin shortcut
- In the second terminal window, run
stardust-xr-server. The display of the headset should switch from grey to black
- In the third terminal window, run
- If all goes well you should have 6DoF and hand tracking running in Stardust XR, and you should be able to grab the red pen in front of / above you and draw with it.
I have to use
gravityhere to run
cometwith an offset, as it seems all of my clients like to run about a metre above my head for some reason!