It's no secret there are a lot of "pseudo-3D" technologies out there, but for MS to be announcing it as a truly revolutionary experience, they had to be using something... amazing.
Well, at last - and actually, by chance - I found what they're using: it's a technology called "Time of Flight", and it allows the creation of a real 3D camera!
Forget everything you knew about 3D technologies. This is no "tricked" 3D image analysis or stereoscopic wizardry. I'm talking about real 3D information from a camera, where each pixel represents a well know 3D point in space.
It uses the same principle of SONAR (with sound) and LIDAR (light), emitting a modulated light pulse and waiting for the time it takes for the reflected signal to get back. Unlike those, however, which require mechanical "swiping" methods to create an "image", time of flight cameras use thousands of receptors - each pixel acts as a tiny range finder - allowing to capture a full 3D image in real time speeds.
And, as each pixel is tuned to look only for a specific modulated light spectrum, it doesn't require - nor is affected - by other light sources.
In the end you get 3 images: one where each pixel intensity represent the distance to that point, one showing a "grayscale" image of the scene, and one other showing a "distance accuracy probability" that tells you how "certain" the camera is of each pixel measurement.
Câmara ToF da MESA Imaging
This kind of technology has therefore nothing to do with "simple" image processing done via regular webcams like the EyeToy. Stuff that is very hard to do with a regular camera (like recognizing overlapping moving objects) suddenly becomes possible, in a much easier way.
The only "issue" is that these cameras currently go for well over $5000 USD (though it's quite feasible to make it a lot more affordable once you built it in mass scale as MS intends to do with its Project Natal.)
If you think all the projects the Wiimote enabled enthusiasts to do... Imagine what an affordable sub $100 ToF camera would allow us to do!
Sean G. x Project Natal from Matthew Pantoja on Vimeo.
But this is not true at all... Project Natal is not a TOF camera and it has nothing in common with RADAR, SONAR or LIDAR which are both based on TOF principles.
ReplyDeleteThis system works by projecting a pattern of infrared dots onto the imaging area, using a setup of cameras to detect the distortion of the patterns as it is received from a different angle.
"Tricked" 3D image analysis and stereoscopic wizardry is exactly what it is, only well executed and quite fast.
It also has no "modulated spectrum" tuning and can definetely be disturbed by other light sources. Flooding the room with infrared light would void the depth detection. Also, remote controls may distort or "blind" the depth perception in affected areas, as will very dark surfaces.
@Anonymous
ReplyDeleteThis was written *before* the Kinect was launched, and when people were trying to figure out how the "3D" camera would work.
Yes, it would be too good to be true to have a ToF camera inside the Kinect, and they went with the pattern projection option - which is a lot cheaper. Anyway, for the intended purpose, it's more than enough - though I sure would love to see MS someday opt for a true ToF camera.