Our blog

29.10.2014 01:24 | Our blog

How an empty crisp packet can be used to eavesdrop on conversations

Source: theguardian.com
Tags investigations
Research from MIT extracts audio from a silent video of everyday objects including crisp packets and potted plants
.
Çàãðóçêà ïëååðà
.
Researchers at the Massachusetts Institute of Technology (MIT), working in conjunction with Microsoft and Adobe, have developed a method of reconstructing sound from a video of an object – letting them use a crisp packet, glass of water, or potted plant as a microphone.
.
The technology is similar to the laser microphones used by spies around the world to eavesdrop on conversations by measuring minute vibrations in reflective surfaces.
But rather than using expensive, specialist equipment, the researchers were able to extract audio from a high-speed video of everyday objects.
.
In one instance, they even extracted recognisable sound from a video shot on a normal digital camera.
“When sound hits an object, it causes the object to vibrate,” explained MIT graduate student Abe Davis. “The motion of this vibration creates a very subtle visual signal that’s usually invisible to the naked eye. People didn’t realise that this information was there.”
.
By analysing high-speed video of a potted plant, for instance, Davis and the rest of the team were able to extract the recording of “Mary Had a Little Lamb” which was played alongside it.
In another experiment, they were able to recreate the voice of a human reading the lyrics to the song from video of a crisp packet filmed through a glass door.
As well as the obvious applications for surveillance and law enforcement, Davis plans on seeing whether the acoustic properties of objects reveal more information about their internal make up, calling it “a new kind of imaging”.
.
“We’re recovering sounds from objects,” he says. “That gives us a lot of information about the sound that’s going on around the object, but it also gives us a lot of information about the object itself, because different objects are going to respond to sound in different ways.”
Typically, the algorithm that allows them to extract this information requires a camera that records very high-speed video; the frequency of audible sound (the number of vibrations every second) ranges from 20 to 20,000 Hz, and a sound can only be extracted by a camera with a frame rate higher than the frequency itself.
Conventional cameras, with a frame rate of just 60 frames per second, cannot be used to extract anything more than extremely low-frequency sounds.
.
But the researchers came up with a second technique, using the “rolling shutter” typical on digital video cameras, to extract enough information from the video to still recreate high-frequency audio.
Because such cameras don’t save the whole image in one go, but rather record the video line-by-line, far more data can be extracted from it. A simple digital SLR camera, shooting video of a crisp packet at 60fps, can then be used to recreate the audio being played to that packet.
Alexei Efros, a professor of electrical engineering at the University of California at Berkeley, praised the work of the group as “new and refreshing”.
.
“We’re scientists, and sometimes we watch these movies, like James Bond, and we think, ‘This is Hollywood theatrics. It’s not possible to do that. This is ridiculous.’ And suddenly, there you have it. This is totally out of some Hollywood thriller. You know that the killer has admitted his guilt because there’s surveillance footage of his potato chip bag vibrating.”
Site development SeaDesign O2