What we see, we generally believe. We barely notice when we perceive something – our brain registers shapes, colors, and information, and it seems automatic. We then use this information to create meaning. This is especially true when an audio cue leads us to perceive something: We hear a car engine roar and look over to see a car driving by.

New research is showing that sounds not only assist in the recognition of images we see, but may even change what we think we see. When we see something, it’s not just our sight that makes up the image – our brain utilizes a lot of energy, pulling various information from all our senses to compile a complete picture. Meanwhile, it feels like an immediate, seamless process. “One way it does this is by making inferences about what sorts of information should be expected.”

These sorts of fill-in-the-blanks that our brain does help us process information. Our mind “assumes” the missing puzzle pieces to make a complete picture, and hopefully, the image it forms is correct. However, this is only sometimes the case, especially if what we hear isn’t aligned with what we anticipate we’ll see. Researchers conducted three experiments on this.

The first experiment utilized 40 participants. They were each shown figures which contained two different objects “morphing into one another” during various stages of the transformation process (for example, a bird morphing into an airplane).

The researchers simultaneously played one of two sounds – one related to what they were seeing (birds chirping, a plane engine roaring) or one completely unrelated, such as the sound of hammering a nail. Participants then used a sliding scale to show which stage the morphing process was in (did the object look more like a bird or plane?).

Researchers discovered that not only did the responses come much quicker when the participants heard the related sound compared to the unrelated sound, but they aligned their choices more closely with the sound they heard. (If they heard birds chirping rather than a plane engine, they thought the object looked more like a bird than a plane).

Now onto the second experiment. Researchers wanted to take it a step further and see if sounds could influence how people make decisions based on what they think they’re seeing. Now, 105 new participants were put through the same experiment (looking at an image of two objects in mid-transformation). However, this time the sounds were played either while the object morph was displayed in front of them or afterward, while the participants were making their selection about which stage the transformation was in (at which time the image was no longer visible).

Similarly to the first experiment, the introduction of sound while viewing the objects impacted the participants – it made their selections quicker and more accurate. However, the sounds had zero effect on the participants after the fact, when the images were taken away and they were left to recall what they had seen.

The third experiment of 40 participants wanted to test what would happen if the sounds were played before the objects were shown. The theory is that hearing sounds beforehand may “prime” a person into making assumptions about what they see later. However, similar to the second experiment, this experiment showed that playing sounds beforehand did not influence the participants’ selections.

“Taken together, these findings suggest that sounds alter visual perception only when audio and visual input occur at the same time.”