Hi, this is Wayne again with a topic “When Videogames Have No Video: Haptic And Non Visual Games”.
My name is jeff Thompson and I’m going to talk very briefly. I have about 15 minutes to talk about video games, interfaces and possibilities for using other experiences in our projects. Things like sound vibrations, smell taste, all that kind of thing, and rather than do kind of a demo of how to do that stuff. I’D rather present kind of case studies, so some game projects that I’ve been working on over the past few years. Don’T we give you some ideas and I have them all up here so afterwards, if you want to come up and hang out and play games, that would be really awesome. So with that, this maybe goes without saying, but video games are really a almost purely visual kind of experience. In some cases, the result isn’t so super fantastic bonus points to anyone who knows which Atari game this is or where they’re buried, but sometimes they’re an incredibly seductive hyper-real experience.
This is the new Grand Theft Auto and I’d like you to notice. One thing that i found was really amazing, which is the motion blur at the bottom of the screen, which must be done in real time, so it simulates the peripheral vision of you driving down the street, which is crazy and kind of amazing, and those things are Really great, but they they mean that we missed some of the really cool possibilities and experiences within games. They might happen if we were to think about other things other than visuals and they leave out players and users that are visually impaired or would otherwise miss out. On these kinds of experiences, so I’d like to give you a little example of what a game might be like without one of these elements, so this is doom. Of course, I’ve spent way too much time playing this game as a teenager. I think I have every level completely memorized, and here we have doom guy now this is actually his real name is doom.
Guy wikipedia confirms this. Doom guy is going to try to open a door here without a key, so I’ll play a little video here. So we hear this sound as he tries to open the door, so I built a mod of doom.
The only changes that I deleted, the off sound effect from the entire game, so you can play the whole game is normal, except without and here’s what that’s like. So it’s a subtle difference, but it changes. We lose this feedback, we don’t know what’s going on, maybe we forgot what button we need to push to open a door. Maybe there’s something else going on so this kind of integration of visuals and sound effects and all these kinds of things together in games is really interesting, which brings me to a series of games that I’ve been making over the past few years. The the first one I’d like to show you was made in collaboration with a really awesome game maker and artist named Alex Meyers. Together, we started a game jam called games plus plus, where people come together in an art gallery context and make games over a period of eight or ten hours, and we wanted a really cool badges for the game jam last year and instead of doing some kind Of static thing, we wanted to make a game and we looked into little displays, but they were too expensive and then we landed on vibration motors.
So, like the same thing, that’s in your cell phone. We bought a bunch of those and the resulting game. I’Ll play a little video here, so you can see how that works.
So the sound – hopefully you can hear – is that vibrating motor right now it’s playing a secret Morse code intro to the game. But the idea is so we hand etched these they’re built on the 18 tiny 85, which is capable of running a super minimal Arduino code and I’d be happy to geek out about some of the details of that too. If anybody wants to, we end up running all four buttons into one analog, pin to maximize the power of the of that little tiny chip and then we hand hand etched all of these. You have four buttons and, like I said, instead of a display there’s this vibrating motor and with this game we didn’t want to kind of explain exactly what it was.
What the game was about. There’S no goals, there’s no enemies, there’s no rewards. We wanted people to explore and kind of map out this thing and figure out what it was I’ll. Let you in on the secret version, which is the idea, is that you’re in this dungeon you’ve been blinded and you wander around.
We could get that on again. You wander around these different tiles and you feel vibration patterns as you walk through this dungeon level, the idea kind of being like you’re feeling with your hands, feeling the surface you’re on if you’re on sand or bumping into a wall, or things like that. These are some of the levels that Alex designed for the game. No one ever sees these as visuals, but you get kind of the secret view and the different colors are different terrains or textures in this dungeon.
So you start in one spot. You want around the little green square is a respawn, so it’s a little secret thing, you land on it and it takes you back to the spot. You started this. One is really a very mean level, and this is what it ends up.
Looking like in the arduino code, so instead of having to do this by hand and kind of map out the levels with that, i wrote a little processing script that takes alex’s awesome image levels that he made in photoshop and converts it into this number array that The arduino can work with and we’re limited to this very small kind of level because of the memory on the little chip. We also showed this game at a really cool exhibition and conference called vector game art fest last year. That was in Toronto and we showed in an exhibition context with a bunch of other art games and things like that. And we left out pieces of graph paper for visitors.
And so they would sit there and try to map out the level as they played. And hopefully you can see this, it’s one of my favorite drawings from that. So I’d like to talk also today about two other games. These are also non visual games, but rather than be built for the Arduino and electronics, they run on the tablet.
These were created using processing for Android, which is really cool. If you haven’t played around with that, it’s really fun to be able to make apps and run them on your tablet, and there it’s not locked down in the same way. Ios makes it it makes it really easy to build interactive stuff in games and you get cool stuff.
You get your camera, you get the vibration motor in here and you get to speakers, and these games were created with some really generous support from an organization called harvest works which is in New York City. I’M going to zip through this first one – and I have these up here so if you want to come play them, please feel free, they’re kind of hard to explain without actually holding and interacting with it. But um cave is similar to that.
You have been blinded. Dungeon game, where you’re tossed into this algorithmically generated dungeon but you’re. The little player is the white circle and you wander around these different tiles feeling your way. There’S vibration patterns, as you walk around sound effects. Some things like that you’re equipped with a flashlight which you can push down and hold and the flashlight comes out, and you can see just a little further around you. So here you can see the different tiles and there’s a wall off to the right side that you bump into and there’s also respawn points in this and again they’re meant to be not good or bad, so you can see them as being holes that you fall Into or bonus things that you go on to and they make sounds and vibration patterns and they also recharge your flashlight because, as you hold it, the flashlight gets dimmer and dimmer.
So, if you bump into this spot, it gives you more flashlights. But the thing I’m most excited about is stuff like this. This next game was inspired by this guy Daniel Kish, he’s legally blind and he’s able to ride a bicycle by echolocation, which is crazy and amazing. He clicks kind of like a bat and he’s able to hear those that sound bounce off of objects and come back to him and he can ride a bike down the street doing this, which is awesome, and I tried kind of learning how to do this.
There’S no way so this guy is amazing, and so the next thing was to make a game that was inspired by this. It’S called sonar. It’S also for the tablet you’re the little circle again, this little guy and it’s probably hard to see on the monitor. But there’s a field of white dots that move down as you’re, going forward, to show you how fast you’re going you tilt. The tablet like this to go faster or slower side to side to steer so kind of like riding a bike.
The thing you can’t see is that there’s objects that are coming towards you as you’re, going along they’re little circles, but you can’t see them because you have to play by sonar and right now what you’re seeing is the chief mode, which also has a visual representation, But you tap the screen and it really only works on headphones, and you hear this sweep just like sonar on a boat and it goes from all the way in your left ear to all the way in your right here and The Closer. These obstacles are to you the louder that sound gets so you’re hearing it kind of bounce off. It’S really really hard to play.
It’S totally doable. It’S really awesome! You end up like closing your eyes and your mersing yourself in this space and again this is the cheat mode, so you can see here’s two obstacles coming towards you. Here’S one! That’S imminently about to crush you, so I’m almost out of time here. I just wanted to offer this as kind of a go. Do this kind of thing, which is we all make cool stuff, we’re all really excited about making things that other people can use? This is a call to arms to think about the non visual experiences in your projects, whether it be from how it’s designed how it looks, how it works for having more meaningful experiences for sighted users and maybe to extend the availability of your projects to people that Might otherwise not be able to access those things? I have some crazy ideas that I think we would be really cool, maybe some that are not so crazy. Vibration feedback live synthesized sound.
I didn’t mention that the cave game has live reverberation that changes depending on the height of the space here in so you can also hear the space expand and contract a hack of Mike Tyson’s punch out where you feel taste blood in your mouth when he punches. You I’m totally going to make that one all these kinds of things, I’m also working on a vibration shield for the Arduino to make it easier for people to use these kind of vibration, waveforms and things. Hopefully, this will be done for next year’s Maker Faire. I can come back and talk to you. I have some early examples of these up here. Thank you.
Come on I’ll play some games. I’Ve got a bunch of these thanks everybody. You .