Click here to support the Daily Grail for as little as $US1 per month on Patreon
Person of Interest - Control's Office

All-Hearing Eye: Researchers Can Reconstruct Audio From Objects’ Micro-Movements Captured Only on Video

By Gregon

Sometimes even science fiction can’t keep up with technological developments…

Consider the excellent Person of Interest, a television show that over four seasons (the fifth and final season began last night) has had a long and fascinating story arc exploring the impact of artificial intelligences on our world, especially in the context of surveillance and law enforcement. In the final episode of season 4 – which aired around a year ago -, one scene has the character ‘Control’ placing cell phones in a sound-proof box, to stop the artificial intelligence named ‘Samaritan’ from hearing the content of her discussion by hijacking the cell’s microphone.

Turns out that may have been a touch naive, especially coming from someone in her position. Because in real life, a couple of months earlier at TED 2015), researcher Abe Davis demoed software that allowed him and his team to reconstruct audio of an event, purely from video taken (ie. no audio required at all) on an off-the-shelf camera. They did so by creating algorithms that looked for tiny movements in objects in the environment (on the order of micrometres, invisible to the human eye and even a fraction of a pixel). As ‘sound’ is actually air vibration perceived by our ears, the micro-movements captured by the video can be used to reconstruct the air vibrations that caused them.

Subtle motion happens around us all the time, including tiny vibrations caused by sound. New technology shows that we can pick up on these vibrations and actually re-create sound and conversations just from a video of a seemingly still object. But now Abe Davis takes it one step further: Watch him demo software that lets anyone interact with these hidden properties, just from a simple video.

Watch the amazing demonstration below:

While Samaritan would surely employ such a system in its ‘sousveillance’ arsenal, Davis does point out that there are other applications as well, including creating an actual ‘3D model’ of an object and its natural movements, simply by capturing its subtle motion.

Related:

Greg
Editor
Greg is the owner-editor of The Daily Grail, as well as the author of a number of books including Stop Worrying, There Probably is an Afterlife.
Related

7 Comments

  1. Just for the record…….
    Since the 1960’s, it’s been possible to eavesdrop on conversations within a room by bouncing a laser off the window. The laser “pulses” in reaction to the vibrations on the glass caused by sound waves. The returning pulses are amplified and whatever sound is happening in the room can be filtered to remove ambient noise and leave the conversations intact.

    The US Navy played around with lasers for several years, when I was flying with them, as a means of secure communication between aircraft. The idea was that a small laser was mounted on your flight helmet,similar to how night vision scopes are, and this was bounced off either the canopy or an observation pane, or a special plate attached to the fuselage. Your voice was sent over the laser as pulses and decoded on the other end.

    That system worked very well for maintaining secure comms but it could be difficult to maintain a link if there was any sort of turbulence bouncing the aircraft around, and of course it was unuseable in clouds and heavy rain, etc.

    Neat stuff though.

  2. New technology at the fringe

    While Samaritan would surely employ such a system in its ‘sousveillance’ arsenal, Davis does point out that there are other applications as well, including creating an actual ‘3D model’ of an object and its natural movements, simply by capturing its subtle motion.

    I wonder if something along these lines could be used on the Patterson-Gimlin film to get some answers. Certainly, there would be detectable differences between a real “animal” and a guy in a suit.

    1. Only if there was imagery of
      Only if there was imagery of a sasquatch that you assumed to be real for use as a comparison.
      I happen to think the Patterson Gimlin footage is of a real sasquatch and should be used as the base comparison footage for all subsequent recorded imagery. The analyses by kinetic experts is very convincing.

      1. To the contrary…
        To the contrary, I think you could get satisfactory results by comparing the prospective Sasquatch to gorillas as well as someone wearing a suit. The only question is whether the film is of sufficient quality for the computer analysis.

        1. Other than humans there are
          Other than humans there are no other primates that locomote anything like a sasquatch – except the sasquatch. The kinesiologists make a very good case for the P-G film not being a “man in a suit.”

          1. Do the Locomotion
            It is not the macro-movement that I am considering. Fur growing from a body should move differently to fur attached to a suit. In this respect, a gorilla would serve as a reasonable approximation.

          2. Oh, I get it – you wanted
            Oh, I get it – you wanted from the get-go to see if a hair suit is involved. That should be doable with the Gimlin footage versus say some footage of a gorilla, but probably some sort of refined algorithm for hair movement would be further needed.

            What I would like to see – given that quite a few witnesses claim to feel an infrasound effect near sasquatches – is an analysis of micromovements in sasquatch film footage to see if an infrasound signature is present.

This site uses Akismet to reduce spam. Learn how your comment data is processed.

Mobile menu - fractal