The inner 'eyes' of the new Kinect game system from Microsoft's Xbox uses different cameras and infrared lights to bathe players and capture their movements. (Courtesy Microsoft)

---

This November, Microsoft will launch Kinect, the latest addition to its Xbox 360 gaming system, with the hope of stealing the crown from Nintendo's Wii console as well as the Sony motion-gaming platform that is also coming out this fall.

The difference is that Kinect requires no hand-held controller, instead using a specially designed camera system to track players' movements, gestures and voice commands.

The goal is not to replace the system's core gaming functionality — the option to use a traditional controller remains — but to make the 360 more accessible to casual players and non-gamers.

"For lots of people, the controller is a barrier that prevents them from getting onto our console and having fun," explains Kudo Tsunoda, the creative director for Kinect, during a demonstration at Microsoft's head office in Redmond, Wash. last week.

Unlike other motion controllers, "you don't have to read any instructions. You don't have to learn any button configurations. You can just jump in and play."

Hardware required

Of course, while this may sound simple, the system's motion and voice-control capabilities require some particularly impressive hardware — including a complicated audio-video subsystem that can identify both a player's position and orientation in space.

Kudo Tsunoda, Microsoft's creative director for the new Kinect system, demonstrates the 'jump and play' capabilities of Xbox's newest toy. (Courtesy Microsoft)

To track a player's movement, a laser projector first bathes the room in invisible light. A secondary infrared camera detects the subtle differences in the way this light hits the scene, and produces a 3D depth map with the data.

At the same time, a third RGB camera provides real-time colour imagery for in-game pictures and facial recognition.

Combined, these cameras allow Kinect to "see" players and separate them from other objects in the surrounding room.

The challenge is teaching the device to identify the movements and gestures that the players' bodies make.

PlayStation's Move, which is coming out in September, and Nintendo's Wii remote rely heavily on accelerometers and gyroscopes to accomplish this task. Kinect, however, uses software algorithms and artificial intelligence to track up to 46 skeletal points on a human frame.

"When we started, you needed to input your weight, your height and shape," recalled Dr. Ilan Spillinger, vice president of hardware and technology for Microsoft's interactive entertainment business. "We do it all now with artificial intelligence. We take the cameras and the depth map, and we merge that data together."

Motion pictures

Members of Kinect's development team travelled to Hollywood to gather massive amounts of motion capture data — the same type used in the blockbuster films that rely on computer-generated imagery.

Those results were reduced to a set of 200 distinct or recognizable poses that help Kinect identify a player's actions.

However, optical data isn't the only method through which Kinect tracks a player's position in space. The device relies on four distinct microphones that can be used to detect and identify a speaker in the room — helping pinpoint his or her location.

A technique called "beam-forming," similar in concept to a submarine's sonar system, measures how quickly each of the four microphones registers a human voice.

The difference in time can then be used to establish that player's approximate position in the room and, in some cases — during a video chat, for example — allows Kinect's built-in motor system to reposition the camera accordingly and follow you around like a souped-up web cam.

But while this process is simple in theory, difficulties arise when Kinect tries to identify voices in a noisy environment. So it has been taught how to compensate.

Similar to how noise-cancelling headphones work, this type of audio processing allows the Xbox to recognize voice commands even in a noisy home theatre environment, regardless of whether a game or movie is being played.

A natural interaction

That's not to say the system isn't without its quirks.

Microsoft's Tsunoda (right)and a visiting tech writer tackle the new Kinect gaming system. (Matthew Braga)

One of the voice-controlled demos we saw seemed unwilling to consistently recognize a particular speaker's voice during video playback, and multiple voices can't always be distinguished from one another either.

As well, Microsoft has announced that only English will be supported on the launch, though other languages, dialects and accents will be supported later on through a software update.

Regardless, the absence here of a hand-held controller is obviously a big selling point, the company hopes, and could prove a compelling option for Xbox owners looking to get into the motion-gaming experience.

The promise of easy accessibility and "natural" interaction is what Microsoft hopes will finally introduce Xbox to the non-gaming, casual market.

"When you think about how most people interact with other humans, you're using your voice and body language as a means of communicating," says Tsunoda. "And now those are ways with which you can actually interact with your Xbox."

Sony will release its own motion-controller, PlayStation Move, Sept. 19. A complete bundle, including a game, motion controller and the PlayStation Eye camera, is to retail for under $99 US

Microsoft's Xbox Kinect motion-controller, meanwhile, will go on sale in North America Nov. 4 for $149.99 US, with a console bundle available at $299.99 US.