Category Archives: VR / AR / XR

Apple Vision Pro – the wait is over; the wait begins

Posted on by

I’ve been waiting patiently for Apple to reveal their headset for at least 5 years now, and as of WWDC 2023, that wait is finally over. A couple of weeks later (last Tuesday), the developer tools dropped, and I finally got a chance to start working with the frameworks and applications that Apple has created for making content on their new headset.

I’ll be eagerly exploring these frameworks and playing with this tech at least until the headset is covering my eyeballs, and (hopefully) for years after. There’s literally so much here that they devoted over 40 (admittedly short) WWDC talks to the topics, (with some overlap, sure, but not as much as you’d think). I have done a fair bit of VR development in the past, and while it’s been a few years, (and I’m sure there’s been some improvement in that time), to me, Apple’s APIs seem like the most flushed-out and easy-to-use way to develop 3D applications that I have ever seen. The only downside I can imagine is the vast amount of new frameworks to learn. There are seemingly a lot of ways to approach the same problem, or similar problems anyway, and I could imagine that being daunting for a lot of people. Fortunately, the documentation is quite good.

I imagine I’ll be writing about this topic a bunch in the next few months, and for my first post on the topic I’m going to just document some of my findings about the coordinate system in visionOS. I’ll jump right in.

position coordinates in immersive space

So I’ve been doing some programmatic drawing experiments on visionOS, and as anyone who has ever drawn things on a screen knows, one of the first things you need to know is:

Where is the center of your drawing area, or x: 0, y: 0, z: 0?

In short, when you open an Immersive Space in visionOS, 0,0,0 is positioned directly beneath you.

While the immersive space remains open – even while you move around within the space – 0,0,0 doesn’t move. But if you close the immersive space, and re-open it, 0,0,0 will have moved to a new location and rotation, again directly beneath your feet, and rotated to face the same way you are facing.

So, because the coordinate space will always appear in the same orientation as you, the following are always true (when you first open the space):

  • -x is to your left, and +x is to your right
  • -y is down, while +y is up
  • -z is forward, and +z is backward (behind you)
    I’m not sure I love that last z-direction decision, but it’s how it works.

Note that animating a transform is as easy as this code snippet:

.gesture(TapGesture().targetedToAnyEntity().onEnded { value in
    var transform = value.entity.transform
    transform.translation += SIMD3(0.1, 0, -0.1)
    value.entity.move(
        to: transform,
        relativeTo: nil,
        duration: 3,
        timingFunction: .easeInOut
    )
})

That example is from the WWDC video Develop your first immersive app.

I have a long list of things to explore next, but at some point in the near future I’ll be exploring drawing and transforms and coordinate spaces in what apple is calling “volumes”. (Volumes are 3D windows, or bounded boxes, essentially.)

Foss XR 2022

Posted on by

Around the beginning of October, I found out that there was going to be an open source XR conference (called FOSS XR) in my neighborhood. I attended, and took some notes. I posted some of my notes in the MSPGameDev slack, and Zach, who runs our local VR & HCI meeting, asked if I would give a short recap of my impressions. I put some slides together, which I’ll embed below the video. There were some technical difficulties getting started, so the beginning of the video is actually as I’m getting into my second or third slide.

Nintendo Labo VR reviewed

Posted on by

As a VR developer, and self-professed Nintendo superfan, getting the Nintendo Labo VR kit was definitely a priority. I had the first couple of Labo kits, but I’ll be honest, I never even put together the giant robot one. (I’ll get around to it someday.)

My daughter and I spent an hour or two putting together the initial headset, and then checking out the 16 or so mini-games that are playable with just that. The headset itself feels a lot like google cardboard, but the larger screen size of the Nintendo Switch means the lenses are also really huge.

I think this also means it’s pretty forgiving in terms of eye placement in the headset. I took the kit to our local VR & HCI meeting earlier this week, and passed it around a bunch. Only one person mentioned they were having problems getting it to line up properly.

My daughter and I have probably also spent another 5 to 10 short (1/2 hr to 45 minute) sessions putting together various other accessories and playing their associated minigames. For some reason that felt quite a bit more satisfying than the original Labo kit, I think maybe because I have a natural tendency to want to only spend a minute or two at a time in 3DoF VR, and so the games feel like they are the appropriate length. And it helps that there are a lot of different experiences.

Additionally, I don’t know how to articulate this exactly, but I cannot stress enough how cool it feels to build your own VR accessory. In a way it feels like switching attachments in Budget Cuts or Cosmic Trip, but IN REAL LIFE, and you build the attachments yourself!

The camera attachment alone was easily an hour of assembly, and the underwater game you play with it (basically just pointing the camera at things and taking pictures) was pretty engaging for a good length of time. We also put together the blaster, (probably more like a 2 or 4 hour activity in total) and I think my daughter would have easily spent at least that long playing the alien shooter game it enables if I’d let her.

In general, I’ve found playing with someone else is a nice way to go about it, because, while in Build mode, it’s nice to have one person folding / assembling the cardboard, and the other person manning the forward button on the switch. And when playing, sessions longer than a minute or two definitely start to make me feel queasy. My daughter doesn’t seem to have that problem, but I’d still like to limit her usage, so taking turns makes sense.

So… I just this morning finished checking out the VR modes in Zelda and Mario Odyssey.

My short review of those: super underwhelming.

The camera still rotates around your character in Zelda, so moving my head at all made me feel pretty sick. You can still control the camera with the right stick, so it was playable if I kept my head still. After like a minute or two, my arms were tired, so I detached the joycons and played with my head tilted back and the Labo VR resting on my face. Made me wish for like a second it came with a headstrap, but it would be terrible, so makes sense that it doesn’t.

The edges of the screen had some chromatic aberration, which was most noticeable in the menus, which float in screen space, so you can’t turn your head to see the edges. Text was hard to read as well (although not as bad as I’d expected).

The whole game is playable in VR, so that’s something. But because the camera is essentially the same as in the normal game, it’s not like you can peer around and find new stuff. The only real novelty is stereoscopic 3D.

Mario wasn’t much better. Their game has the camera at a fixed point, and you HAVE to rotate the headset to look around. This doesn’t make me feel sick, really, but feels like a super low-rez way to play what is otherwise a good looking game. (For some reason, the low rez in Zelda didn’t bother me at all, but I wasn’t checking it out for very long…)

The gameplay in the mario levels (because that’s what they are, new smallish 3D levels, where the camera only rotates from a fixed point) was collecting musical instruments to give to the musicians scattered throughout. To get each instrument was basically the musical-note-fetch-quest that triggers by collecting the first note, (and then a bunch more notes appear to collect before a timer makes them disappear again). This is a mechanic already in the base game, typically with stars as a reward, and was rather disappointed it wasn’t a new game mode entirely. Especially because those are some of my least favorite types of stars to collect. And it’s worth noting you don’t even get stars for them here. You load up these new levels from the main menu, before you even pick a save slot.

You also have the option of playing the new levels without VR. But in that mode you CANNOT change the camera other than using the Nintendo Switch’s accelerometer. I was disappointed with the first level in VR, so I thought I’d try out the second one without it, and it definitely felt worse to me having to rotate the switch around to look anywhere, but without it attached to your face.

Overall, I am VERY pleased with my Nintendo Labo VR purchase. I just wouldn’t recommend getting it to play Zelda or Super Mario Odyssey. But there is a ton of content in the Labo kit itself. I still haven’t built the Elephant or Swan attachments, and I feel like I have a lot more content to explore even without them! I’m definitely looking forward to spending more time with Labo VR.

Vive Hockey

Posted on by

I did a quick (~15 minute) presentation at the last MN VR & HCI meeting earlier this week about the hockey game I worked on with/for Visual and Hiway Credit Union. When I polled the audience before launching into it, almost nobody in the room was a developer (it was one of the sparser attended VR meetups in recent memory), so I rushed through most of the technical bits. (Which was the focus of the whole talk!)

Anyway, here are the slides, for posterity:

GDC / VRDC 2017 Recap

Posted on by

I gave a 7 (-ish) minute presentation tonight at our local VR & HCI meeting. I was one of three folks invited to share their GDC / VRDC experiences. Here are the slides:

Additionally, I was contacted today by a student looking for more information about my experiences in the local game industry. If you are interested in that sort of thing, I’ve posted the transcript below.

Continue reading →