Watching What You Say

JUNE 1, 2016

We don’t really think about sound all that much, and when we do, we rarely think of it as something physically traveling through space. But that’s really how sound works. Sound waves are pressure waves — they cause vibrations in the area around them. When we hear those sounds, it’s because those vibrations strike our eardrums, some more stuff happens, and ultimately, our brains translate that into meaningful (or meaningless) noise.

But when we’re having conversations, those pressure waves don’t just go into our ear. They bounce around the room, hitting against everything from the bag of potato chips on the table to the windows and doors. So it logically follows that, if your conversations are causing glass windows to vibrate, maybe those vibrations can be seen from the outside — by people who would not otherwise be able to hear you conversations.

Actually, there’s no “maybe” about it.

In May of 2011, the United States raided a compound in Pakistan, killing terrorist mastermind Osama bin Laden. A few days later, a global security expert named John Pike talked to NPR about the raid. The NPR host, Melissa Block, noted that U.S. authorities “knew 22 people were inside that compound, including someone they describe as an adult male who they say never stepped into view.” Sensing the obvious conundrum, Block asked Pike how can we see someone who never stepped into view? Pike explained that we don’t have to see the person — we can see the imprint of his sound waves:

A conversation in a room is going to cause windows to vibrate. If you shine a laser beam on those windows, you can detect those vibrations, and using voice identification, you can figure out how many different voices are speaking in each of the rooms of the compound.

Per Pike, the U.S. government was able to do just that, likely by stationing the laser-equipped listening team in a tall building nearby the Bin Laden compound. And to make things even cleaner, Pike continued, this came with almost no risk of being detected: “the laser beam is not visible to the human eye.” American intelligence could “see” the conversations; the terrorists couldn’t see the listening device.

The ability to “see” conversations may not be limited to intelligence operations, either. In 2014, researchers at MIT announced that you may not even need a laser to “see” sounds — a camera, combined with some new MIT software, can do the trick. As seen in this video, the MIT team placed a bag of potato chips (and other objects) near people singing and having conversations. The researchers recorded the bag of chips, and, while it seemed to be perfectly still to the naked eye, the camera was able to pick up some slight movements. Those movements, the MIT team theorized, were caused by the pressure waves from the nearby noise. The MIT lab collected data about those movements and was able to translate those movements into sound, coming pretty close to approximating the conversations etc. that the potato chip bag “heard.”

That technology is (thankfully?) still a long way off, but it’s getting there — if you fast forward to the end of that video (or just click here), you’ll see that the MIT team doesn’t need a fancy camera to do its magic — a regular smart phone camera will do the trick. The audio derived from those cameras is far from perfect, but it’s still a reasonably good approximation.