“When sound hits an object, it causes the object to vibrate,” says Abe Davis, a graduate student in electrical engineering and computer science at MIT. “The motion of this vibration creates a very subtle visual signal that’s usually invisible to the naked eye. People didn’t realize that this information was there.”
An algorithm was developed by researchers at MIT, Microsoft and Adobe to reconstruct these subtle visual signals. They’ve successfully recovered intelligible speech from vibrations of a potato chip bag through soundproof glass. Other successful surfaces include the surface of a glass of water, a sheet of aluminum foil and the leaves of a potted plant.
A new way of looking at reading lips
The team behind this research alongside Davis includes Bill Freeman and Fredo Durand, both MIT professors, Neal Wadhwa, a graduate student under Freeman; Michael Rubinstein of Microsoft; and Gautham Mysore from Adobe Research.
The applications are numerous, most obviously in law enforcement, surveillance and forensics. The researchers are assured that even on a conventional 60 frames-per-second recording, there may be enough fidelity to identify the number of speakers, the gender of the speaker, and even the speaker’s identity.
The scientists, however, are more excited about the possibility of a new form of imaging. Essentially, this algorithm recovers sound from objects which tells you how the object interacts with sound which in turn gives you structural information about the object. Davis believes this might lead to an aural characterization of objects, the nerd.
With any scientific breakthrough though, the real excitement comes when technology hits the market. With public and free market interaction, the technology is often repurposed by other creatively driven or even profit-driven actors into areas the creators would never have fathomed. Considering that this literally translates information from one sensory medium to another, this development could have more potential excitement than others in maturing its possible applications.
[Source, image: MIT]