Category Archives: VR / AR / XR

vision pro game in progress

Posted on by

I’m timeboxing the creation of this blog post, so I can continue to work on my game for the apple vision pro. I’ve had the thing for a week now, and in the weeks leading up to the launch date, I began working on a port of a game I’ve worked on previously (but never released) called Puzzle Prison.

I have planned (for quite a while, really) to write a blog post about how and why I never launched Puzzle Prison, but I’ll go into it briefly here. It was a game I developed for originally for Google Cardboard. I actually showed it at an event at GDC one year (2016, maybe?), and at numerous events locally around the Twin Cities. (Various VR & HCI events and meetings, Twin Cities Maker Faire, in the lobby at a TEDx event, and probably other events too.)

The game is fairly simple. It’s a SameGame variant, with the twist that the game is played on four walls, one in each direction. It is a stupidly simple way to bring a 2D grid-based game into the third dimension, and despite having lots of ideas for how to make the game better than what I describe (I implemented a few of them), the simplest game ideas are often the best ones, and the base game idea was probably the best version of it.

Here’s a trailer I put out there at some point:

One reason I never launched the thing was that new platform development kept being more compelling than launching for the current platform. By the time the game was polished enough to release for Google Cardboard, I was doing a lot of “real” (3DoF) VR development, mostly for Vive, and I decided that I thought Google Cardboard was essentially dead, and I would wait to release it when the steam version was finished. Then I had similar thoughts about SteamVR, and wanted to release it for the Oculus Quest. I really should have just launched it on iOS and Android and been done with it.

Anyway, fast forward a bunch of years to this year’s WWDC, and I’m ecstatic to be able to work on a game for Apple’s new headset in Swift! And the APIs look great! I spent some time looking through my various VR game ideas, and nothing really jumped out at me as something as easy to make as Puzzle Prison. The project has another big plus going for it, and that’s the fact that I already have a bunch of awesome audio assets that I’d love to use and include.

Okay, so it’s a week after the headset launched. Two weeks ago, I thought I’d have an app out by now. What’s the holdup?

First thing, the game is playable. I’m tempted to launch a first version as/is, and improve it incrementally. But it’s missing a lot of stuff, and for one thing, I still haven’t implemented those aforementioned audio files.

But there’s another issue I’m working through, and weirdly, it’s related to game design. The original game has what I think are pretty awesome “block breaking” animations. (I’ll get to how this is relevant in a second.) I actually think they may be some of the best animations I’ve ever created in a game. (To be honest, it might be the only animation I’ve ever created that wasn’t directly necessary for a game’s design.) The game without them is totally fine, and playable… except… they take some time. A second or two. And at some point I implemented a feature in the old version of the game where there is a multiplier. If you break more than one group at a time, it increments a multiplier, and the second group is worth more points. But because the game currently has no animation, the scoring isn’t the same as the old game.

There are other effects that I think are more important than the block breaking effect. Namely showing you when and where the game is in danger of ending, as well as an effect for when the game does actually end. Both are not just cosmetic, but have important UX impact. So they may get my attention first, but I’ll get to those block breaking animations eventually.

Apple Vision Pro – the wait is over; the wait begins

Posted on by

I’ve been waiting patiently for Apple to reveal their headset for at least 5 years now, and as of WWDC 2023, that wait is finally over. A couple of weeks later (last Tuesday), the developer tools dropped, and I finally got a chance to start working with the frameworks and applications that Apple has created for making content on their new headset.

I’ll be eagerly exploring these frameworks and playing with this tech at least until the headset is covering my eyeballs, and (hopefully) for years after. There’s literally so much here that they devoted over 40 (admittedly short) WWDC talks to the topics, (with some overlap, sure, but not as much as you’d think). I have done a fair bit of VR development in the past, and while it’s been a few years, (and I’m sure there’s been some improvement in that time), to me, Apple’s APIs seem like the most flushed-out and easy-to-use way to develop 3D applications that I have ever seen. The only downside I can imagine is the vast amount of new frameworks to learn. There are seemingly a lot of ways to approach the same problem, or similar problems anyway, and I could imagine that being daunting for a lot of people. Fortunately, the documentation is quite good.

I imagine I’ll be writing about this topic a bunch in the next few months, and for my first post on the topic I’m going to just document some of my findings about the coordinate system in visionOS. I’ll jump right in.

position coordinates in immersive space

So I’ve been doing some programmatic drawing experiments on visionOS, and as anyone who has ever drawn things on a screen knows, one of the first things you need to know is:

Where is the center of your drawing area, or x: 0, y: 0, z: 0?

In short, when you open an Immersive Space in visionOS, 0,0,0 is positioned directly beneath you.

While the immersive space remains open – even while you move around within the space – 0,0,0 doesn’t move. But if you close the immersive space, and re-open it, 0,0,0 will have moved to a new location and rotation, again directly beneath your feet, and rotated to face the same way you are facing.

So, because the coordinate space will always appear in the same orientation as you, the following are always true (when you first open the space):

  • -x is to your left, and +x is to your right
  • -y is down, while +y is up
  • -z is forward, and +z is backward (behind you)
    I’m not sure I love that last z-direction decision, but it’s how it works.

Note that animating a transform is as easy as this code snippet:

.gesture(TapGesture().targetedToAnyEntity().onEnded { value in
    var transform = value.entity.transform
    transform.translation += SIMD3(0.1, 0, -0.1)
    value.entity.move(
        to: transform,
        relativeTo: nil,
        duration: 3,
        timingFunction: .easeInOut
    )
})

That example is from the WWDC video Develop your first immersive app.

I have a long list of things to explore next, but at some point in the near future I’ll be exploring drawing and transforms and coordinate spaces in what apple is calling “volumes”. (Volumes are 3D windows, or bounded boxes, essentially.)

Foss XR 2022

Posted on by

Around the beginning of October, I found out that there was going to be an open source XR conference (called FOSS XR) in my neighborhood. I attended, and took some notes. I posted some of my notes in the MSPGameDev slack, and Zach, who runs our local VR & HCI meeting, asked if I would give a short recap of my impressions. I put some slides together, which I’ll embed below the video. There were some technical difficulties getting started, so the beginning of the video is actually as I’m getting into my second or third slide.

Nintendo Labo VR reviewed

Posted on by

As a VR developer, and self-professed Nintendo superfan, getting the Nintendo Labo VR kit was definitely a priority. I had the first couple of Labo kits, but I’ll be honest, I never even put together the giant robot one. (I’ll get around to it someday.)

My daughter and I spent an hour or two putting together the initial headset, and then checking out the 16 or so mini-games that are playable with just that. The headset itself feels a lot like google cardboard, but the larger screen size of the Nintendo Switch means the lenses are also really huge.

I think this also means it’s pretty forgiving in terms of eye placement in the headset. I took the kit to our local VR & HCI meeting earlier this week, and passed it around a bunch. Only one person mentioned they were having problems getting it to line up properly.

My daughter and I have probably also spent another 5 to 10 short (1/2 hr to 45 minute) sessions putting together various other accessories and playing their associated minigames. For some reason that felt quite a bit more satisfying than the original Labo kit, I think maybe because I have a natural tendency to want to only spend a minute or two at a time in 3DoF VR, and so the games feel like they are the appropriate length. And it helps that there are a lot of different experiences.

Additionally, I don’t know how to articulate this exactly, but I cannot stress enough how cool it feels to build your own VR accessory. In a way it feels like switching attachments in Budget Cuts or Cosmic Trip, but IN REAL LIFE, and you build the attachments yourself!

The camera attachment alone was easily an hour of assembly, and the underwater game you play with it (basically just pointing the camera at things and taking pictures) was pretty engaging for a good length of time. We also put together the blaster, (probably more like a 2 or 4 hour activity in total) and I think my daughter would have easily spent at least that long playing the alien shooter game it enables if I’d let her.

In general, I’ve found playing with someone else is a nice way to go about it, because, while in Build mode, it’s nice to have one person folding / assembling the cardboard, and the other person manning the forward button on the switch. And when playing, sessions longer than a minute or two definitely start to make me feel queasy. My daughter doesn’t seem to have that problem, but I’d still like to limit her usage, so taking turns makes sense.

So… I just this morning finished checking out the VR modes in Zelda and Mario Odyssey.

My short review of those: super underwhelming.

The camera still rotates around your character in Zelda, so moving my head at all made me feel pretty sick. You can still control the camera with the right stick, so it was playable if I kept my head still. After like a minute or two, my arms were tired, so I detached the joycons and played with my head tilted back and the Labo VR resting on my face. Made me wish for like a second it came with a headstrap, but it would be terrible, so makes sense that it doesn’t.

The edges of the screen had some chromatic aberration, which was most noticeable in the menus, which float in screen space, so you can’t turn your head to see the edges. Text was hard to read as well (although not as bad as I’d expected).

The whole game is playable in VR, so that’s something. But because the camera is essentially the same as in the normal game, it’s not like you can peer around and find new stuff. The only real novelty is stereoscopic 3D.

Mario wasn’t much better. Their game has the camera at a fixed point, and you HAVE to rotate the headset to look around. This doesn’t make me feel sick, really, but feels like a super low-rez way to play what is otherwise a good looking game. (For some reason, the low rez in Zelda didn’t bother me at all, but I wasn’t checking it out for very long…)

The gameplay in the mario levels (because that’s what they are, new smallish 3D levels, where the camera only rotates from a fixed point) was collecting musical instruments to give to the musicians scattered throughout. To get each instrument was basically the musical-note-fetch-quest that triggers by collecting the first note, (and then a bunch more notes appear to collect before a timer makes them disappear again). This is a mechanic already in the base game, typically with stars as a reward, and was rather disappointed it wasn’t a new game mode entirely. Especially because those are some of my least favorite types of stars to collect. And it’s worth noting you don’t even get stars for them here. You load up these new levels from the main menu, before you even pick a save slot.

You also have the option of playing the new levels without VR. But in that mode you CANNOT change the camera other than using the Nintendo Switch’s accelerometer. I was disappointed with the first level in VR, so I thought I’d try out the second one without it, and it definitely felt worse to me having to rotate the switch around to look anywhere, but without it attached to your face.

Overall, I am VERY pleased with my Nintendo Labo VR purchase. I just wouldn’t recommend getting it to play Zelda or Super Mario Odyssey. But there is a ton of content in the Labo kit itself. I still haven’t built the Elephant or Swan attachments, and I feel like I have a lot more content to explore even without them! I’m definitely looking forward to spending more time with Labo VR.

Vive Hockey

Posted on by

I did a quick (~15 minute) presentation at the last MN VR & HCI meeting earlier this week about the hockey game I worked on with/for Visual and Hiway Credit Union. When I polled the audience before launching into it, almost nobody in the room was a developer (it was one of the sparser attended VR meetups in recent memory), so I rushed through most of the technical bits. (Which was the focus of the whole talk!)

Anyway, here are the slides, for posterity:

GDC / VRDC 2017 Recap

Posted on by

I gave a 7 (-ish) minute presentation tonight at our local VR & HCI meeting. I was one of three folks invited to share their GDC / VRDC experiences. Here are the slides:

Additionally, I was contacted today by a student looking for more information about my experiences in the local game industry. If you are interested in that sort of thing, I’ve posted the transcript below.

Continue reading →