Educational gameplay prototypes

As part of the process for making Wired, I worked with some students over the summer to create a load of gameplay prototypes.

Nearly all of these games took one person 2 weeks to make (though a couple took longer). They all have something interesting about them and are also all flawed to some extent. However, each prototype showed enough to know whether the core idea would work and where the potential risks lay.

You can play the actual games here:
http://i-want-to-study-engineering.org/gameslist/

Wonderbook

I worked on a number of camera-based games for Playstation during my ten years at Sony.

This video demonstrate the book tracker my team developed. The player holds a real cardboard book in their hands while the PlayStation camera looks at them. On the TV, the book is transformed into a magic spell book. There are a number of systems working together to pull off the illusion including: Book tracking, hand segmentation, PS Move controller tracking and motion detection. Note that the PSEye camera is a fairly standard web camera and doesn’t have a depth sensor.

In this link you can see an overview presentation of how the book tracker part worked. The photos in the presentation show Tom Lucas-Woodley, one of the senior engineers who poured his soul into this to get it working:
book-tracking_reduced.pdf

You can see adverts for the actual products the technology went into here:

SLAM

SLAM stands for Simultaneous Localisation and Mapping. This was a system my team at Sony developed for mobile gaming devices. You look around an environment with the camera and it builds a 3D map. You can then use this map to place virtual objects in the environment or indeed warp the environment itself.

The table warping demo used the front and back touchscreen of the PlayStation Vita to interact.

While the team I managed developed this technology, I didn’t do any of the technical work myself. This was primarily developed by Antonio Martini, Simon Hall and Andrew Stoddart.

Optical flow based UI

I created some interesting new user interface widgets using a camera. Most of the camera-based games we created did not require the player to hold a controller to play them so it was important that we could navigate menus without a controller too.

The first part of the video shows a “scroll bar” which could be moved with your hand. This suffered from the problem that it could be accidentally moved by someone standing behind it. The second part of the video is an attempt to get around this problem with a “dial controller” which only responds to circular motion.

Head Tracking Game Mechanic

Many camera based games tend to be quite unskilled. This was an attempt to make a game mechanic which required a lot of practice to master.

The game mechanic worked rather well. However, we never really got the head tracker reliable enough to use it. It isn’t enough to just track a head, you need to be able to detect when it has erroneously detected a spectator standing in the background too.

EyePet

Before working on Wonderbook, I worked on EyePet. An augmented reality virtual Pet game for PlayStation 3.

The game was shipped with a “Magic Card” which was used to move objects in the game work around in 3D. This was before the PS Move controller was developed. The Magic card  uses the fiducial marker system which was later used in Wonderbook. Since this was not tolerant to occlusion, the card had to be physical designed so it was always held without the players hands covering the marker.