Now Razorfish's Emerging Experiences team has taken a pair of Kinects and turned them into KinectiChord, a 3D control interface for music making – and it looks rather special, as you can see from the video below.
"This is one of the things that just came out of our labs," says Razorfish group creative director Luke Hamilton. "We're on the advisory council for the Kinect for Windows team, so we get advanced versions of things. So we've been playing with some of the new software to do better tracking. "We're experimenting with the theory of 'anything that can be a pixel will be a pixel in the future.' As surfaces become interactive, what can we do with them?"
The KinectiChord uses a twin-Kinect setup. One of the cameras faces the audience, monitoring who's standing in front of it – "We can do things like measure gender and age if we want – maybe we'll change the music out as a result of the kind of audience that's in front of it," says Hamilton. Walk closer to the screen, and it'll start tweaking the visuals to grab your interest – and fire up the second Kinect, which is trained on the KinectiChord's flexible screen. It's used to measure where you're pressing on the screen, and the force with which you're pushing – "That's used to affect audio tracks, volume, panning, EQ, so that you can play with the music."
"It's still in the beta phase," notes Hamilton. "We have a lot of plans for it; you're actually going to be able to DJ on this in the future." And one of those plans is to take advantage of the Xbox One's next-gen Kinect sensor. "The KinectiChord is using the old hardware, and we can't do really precise interactions with it, it's more big gesture pushing," says Hamilton. "In the future you'll be able to just push a finger through and we'll be able to detect it – that's how high the resolution is on the depth measurement."
Okay, we might not all have the room to set up a pair of Kinect sensors and a flexible screen in our homes – but Razorfish's beat is in retail and advertising. So expect to see the KinectiChord gracing a shop window near you soon – hot on the heels of Alan Sugar's Minority Report-style Amscreen ad hoardings.