THEATER OF THE MIND
DIGITAL PROGRAM
Science Explanations
FUNERAL: TRITONE PARADOX
The music at David’s funeral is your first introduction to the concept that your reality is not my reality!
The basic pattern that produces this illusion consists of two computer-produced tones that are related by a half-octave. (This interval is called a tritone). When one tone of a pair is played, followed by the second, some people hear an ascending pattern. But other people, on listening to the identical pair of tones, hear a descending pattern instead. This experience can be particularly astonishing to a group of musicians who are all quite certain of their judgments, and yet disagree completely as to whether such a pair of tones is moving up or down in pitch.
Diana Deutch’s Tritone Paradox has another curious feature. In general, when a melody is played in one key, and it is then transposed to a different key, the perceived relations between the tones are unchanged. The notion that a melody might change shape when it is transposed from one key to another seems as paradoxical as the notion that a circle might turn into a square when it is shifted to a different position in space.
But the Tritone Paradox violates this rule. When one of these tone pairs is played (such as C followed by F#) a listener might hear a descending pattern. Yet when a different tone pair is played (such as G# followed by D), the same listener hears an ascending pattern instead. (Another listener might hear the C-F# pattern as ascending and hear the G#-D pattern as descending.)
The tones that are employed to create the Tritone Paradox are so constructed that their note names (C, C#, D and so on) are clearly defined, but they are ambiguous with respect to which octave they are in. For example, one tone might clearly be a C, but in principle it could be middle C, or the C an octave above, or the C an octave below. This ambiguity is built into the tones themselves. So when someone is asked to judge, for example, whether the pair of tones D-G# is ascending or descending in pitch, there is literally no right or wrong answer. Whether the tones appear to move up or down in pitch depends entirely on the mind of the listener. (Ambiguous tones such as these were used by Roger Shepard and Jean-Claude Risset to create illusions of endlessly ascending or descending pitches.)
The way that any one listener hears the Tritone Paradox depends on the names of the notes that are played. The musical scale is created by dividing the octave into twelve semitone steps, and each tone is given a name: C, C#, D, D#, E, F, F#, G, G#, A, A# and B. The entire scale, as it ascends in height, consists of the repeating occurrence of this succession of note names across octaves. So when you move up a piano keyboard in semitone steps beginning on C, you go first to C#, then D, then D#, and so on, until you get to A#, then B, and then C again. At this point you have reached an octave, and you begin all over, repeating the same series of note names in the next octave up the keyboard.
Because all Cs sound in a sense equivalent, as do all C#s, all Ds, and so on, we can think of pitch as varying both along a simple dimension of height and also along a circular dimension of pitch class - a term that musicians use to describe note names. So, for example, all Cs are in pitch class C, all C#s are in pitch class C#, and all Ds are in pitch class D.
This strange illusion has implications for the relationship between speech and music. Philosophers and composers have argued for centuries that strong linkages must exist between these two forms of communication. Indeed, many composers, in their search for expressivity, have incorporated into their music features that are characteristic of spoken language. The Tritone Paradox shows that the speech patterns to which we have been exposed can indeed influence how music is perceived.
FUNERAL: CHECKER SHADOW ILLUSION
The shades of grayness in Edward Adelson’s checkerboard are determined not by the actual light waves coming from gray-appearing or white-appearing tiles – these are the same – but by the brain’s best guess about what caused these particular combinations of wavelengths, and – as with “The Blue/Black Dress Phenomenon” – this depends on context. The white-appearing tile is in shadow, the gray-appearing tile is not, and the brain’s visual system has inscribed deep in its circuitry the knowledge that objects in shadow appear darker.
In just the same way that the brain adjusts its perceptual inferences on the basis of ambient lighting, it adjusts its inferences about the shade of the tiles on the basis of prior knowledge about shadows. This is why we perceive the white-appearing tile as being lighter than the shadow-free gray-appearing tile.
This is all completely automatic. You are not – or at least were not – aware that your brain possesses and uses prior expectations about shadows when making its perceptual predictions. It’s also not a failure of the visual system. A useful visual system is not meant to be a light meter, of the sort used by photographers. The function of perception, at least to a first approximation, is to figure out the most likely causes of the sensory signals, not to deliver awareness of the sensory signals themselves – whatever that might mean.
SKULL: GIANT FLASH
Our perceptual experience is not the direct experience of the world around us. Rather, it is the result of a constructive process, in which the brain formulates an ongoing hypothesis about the external world. Visual perception transforms the two tiny movies that constantly dance across the retinal screens on the back of each eye into the rich world we feel we inhabit.
Opsins are biological pigments that reside in the photoreceptor cells of the retina, the cells that are responsible for the first events in the perception of light. Rhodopsin is the most sensitive opsin. It resides in the 120 million rod photoreceptors that sit in each of our retinae, enabling us to see in low illumination.
In this experiment, a flash of light exposes the audience to an afterimage as we create very unusual sensory conditions that allow them to experience the unfolding process of perceptual
When the flash goes off, you see your hand in front of your face, but as you move your hand down to your lap—while leaving your head and eyes fixed forward—you experience rich visual percepts that emerge through the interplay that takes place between our proprioceptive bodily sense and a visual system that is activated in total darkness.
We use red lighting during the initial dark adaptation, because rods have no sensitivity in the red end of the spectrum. The effect depends on the rods that take a while to recover their sensitivity when the light level is reduced. That is the reason for the 7-minute dark adaptation and for the long wait between flashes.
What you see after the flash is called an afterimage, but it is unlike ordinary afterimages—it is a positive version of what you saw during the flash. This is because there is so much light in the flash, it creates a big backlog of activated molecules in the rods. These can only be read out and converted to neural signals at a slow rate and so the same signal keeps being sent until the backlog is cleared. This is happening in the retina where the rod receptors are found, but it is not full color, it is full range grayscale with high contrast and good 3D if the eyes are properly focused and converged on the target. It is not color because there is only one type of rod. Some people report it is purplish or purple green and that is the color we may see from rod activity when it is very strong, but even so it is just shades of this slight color that we see.
When the body motion no longer matches these images, the brain compensates as best it can—in ways that sometimes result in impossible images. The brain is always trying to veto impossible signals — for example, we do not see the many blood vessels overlying our retinas even though they cast very strong shadows. The logic is that any persisting image could possibly be due to sensor errors — miscalibrations of the receptors or bits of stuff overlying the receptors — so something that lasts too long is suppressed. These afterimages are suppressed before their long lasting signal runs out and even faster when there is any reason to doubt the persisting image — for example if it no longer matches the positions of the visible body parts or does not change when the eyes move.
DISCO: MIRACLE BERRIES
Few examples demonstrate the illusory nature of our chemical senses better than Synsepalum dulcificum, otherwise known as the miracle-berry plant. Native to West Africa, this shrub grows a few meters high and produces 2cm red fruit shaped a little like olives or acorns – so far, it may sound rather unassuming and not very miraculous. But the miracle happens when you chew the flesh of the berry.
The flavor is tangy and very mildly sweet, but nothing much to write home about. After you’ve eaten the berry, though, if you then bite a lemon or a lime or drink some vinegar, your mouth will be flooded with sweetness—an explosion of the intense taste of sugar. The sweet taste will linger for several minutes, until the active compound of the berry, a substance known as miraculin, is washed away by your saliva.
On its own, miraculin does nothing. In the normal setting of the mouth, with neutral saliva, the miraculin binds to sweet receptors and simply blocks them, not really stimulating a sensation of sweet. But when the mouth becomes acid, when saliva mixes with something sour, the miraculin binds with salivary proteins and changes its structure, suddenly making it able to trigger these sweet receptors rather than blocking them. A rather remarkable illusion of taste, it is a potent illustration of the loose nature of the relationship between the physical world and our experience of it.
DISCO: MOTION-INDUCED BLINDNESS
Enter the Theater of the Mind disco: a huge rotating mirror ball is built around a central column in the room. The mirror ball rotates slowly and steadily, while light sources hung from the ceiling target the mirror ball. This simple set-up effectively demonstrates two different perceptual illusions—vection—a visually-induced sensation of self-motion—and motion-induced blindness (MIB).
Once you’re positioned in place around the column, underneath the disco ball, you stare out at the illuminated faces hanging opposite you on the wall. Your fellow audience members are stationed in the corners of the room, to the left and right of the mask, surrounded by moving dots. As you stare at the mask, they begin to disappear. Typically within 15–20 seconds, the person’s body starts fading and disappears partly or entirely. If the person starts moving their head or arm, only that part of the body reappears.
How can a live human body disappear? There is not yet consensus on the precise neurological mechanism that is responsible, but several ideas have been proposed over the years.
One suggestion for the cause of MIB has to do with the focus of attention. Perhaps the brain is simply distracted by the changing image and fails to pay attention to static portions of the visual field. In this respect, MIB may be similar to an effect produced when two different images are seen by the two eyes (binocular rivalry). In that case, the brain tends to focus on the image in one (dominant) eye, to the exclusion of the image in the other eye.
There is some evidence that some eye movements are partially suppressed in the presence of background motion. In that case, the reduced eye motion may allow desensitization to occur. However, more recent studies of eye movements in MIB experiments claim that eye movement suppression is not adequate to allow significant desensitization.
Some research has suggested that MIB might be related to a known effect called "perceptual filling-in", which explains why we don't see the blind spots in our eyes. In the absence of sensory input for portions of the visual field, the brain fills in the missing parts with information from surrounding areas.
It has also been suggested that if a static image is inconsistent with the motion of its background, the brain may discount the static image as contrary to the logic of the scene, perhaps akin to how it ignores the blind spot in the eye.
The eye and brain integrate the visual image over time. For images that are moving, this integration produces streaking in visual perception, like the streaking in a long exposure photograph of fast motion. The brain actively suppresses the streaking in order to make better sense of the image. This suppression may account for some or all of the MIB effect. Research has demonstrated that MIB is enhanced at the trailing edge of motion, which supports this idea.
BACKSTAGE: CHANGE BLINDNESS
When you pay attention to something – for example, really trying to see whether a gorilla is out there in the distance – your brain is increasing the precision weighting on the corresponding sensory signals, which is equivalent to increasing their estimated reliability, or turning up their ‘gain’. Thinking about attention this way can explain why sometimes we don’t see things, even if they are in plain view, and even if we are looking right at them. If you are paying attention to some sensory data – increasing their estimated precision – then other sensory data will have less influence on updating perceptual best guesses.
But what about if you’re told that changes are coming? How can we still miss those?
In the video you saw Backstage, a street scene in Glasgow is intercut with black screens—or blinks—each less than a tenth of a second. With each blink, a component of the scene changes: a tree, a car, a sign, the complete facade of a building! In fact, in some 30 blinks, the entire street scene is replaced by another image taken along the same street.
People are good at detecting small changes in what they’re looking at. But with the blinks in this video, the entire field of view changes twice, first to black, then to the new image, and that is enough to make the smaller changes imperceptible. If you know that part of the image is going to change, and you concentrate on that, you will notice the change when it happens, even with the blink. But you still won’t notice all the other changes.
Change blindness shows we only see what we’re paying attention to. It has implications for short-term memory.
BACKYARD: UNCONSCIOUS ADAPTATION
Here we play a simple game of toss! You probably easily succeeded or came close to getting the washers into the center container at first, but once you put on the goggles you realize that things are no longer as easy as they seemed.
These goggles shift your vision 30 degrees to the left, so you likely initially consistently missed the center container by 30 degrees. Over time, you corrected for the prisms and got closer and closer to the target. It became easier, perhaps as easy as it was when you first tried.
When the goggles are removed, however, you probably missed by 30 degrees in the opposite direction, because you learned and adjusted without being consciously aware. This failure reveals how our unconscious adaptation can override our conscious will.
The adaptive unconscious, first coined by social psychologist Daniel Wegner in 2002, is described as a set of mental processes that is able to affect judgment and decision-making, but is out of reach of the conscious mind. This ability to adapt and change is what has kept us and other organisms alive.
ATTIC: EMBODIMENT
When you put on the VR headset, rest back into the chair looking forward, put your feet up on the stool, hold the knobs on the armrests and eventually see a tiny body, different from your own, in the same position, in a starkly different setting than the one you sat your own body down in, you start to feel eerily possessed. Those legs, those arms, even those emotions—to a point—start to feel like your own. This is emphasized when the guide tries to cut the doll’s hair, and you are just as terrified by that scissor blade as the guide’s voice portrays.
The scientists whose work we are employing in this room think that even paintings often personify this phenomena of embodied perception. The viewer, when viewing a painting, is embodied in a POV which gives the painting more power and effect, as one is, in effect, viewing it through the eyes of another. Painting and proprioception have a relationship.
Embodiment is actually more about the body as a reference for outside objects and spaces… and the whole world. It’s more than just a body swap! The embodied process is a technique used to safely meet and digest unprocessed life experience, which then has the ability to lift us up to experiencing higher and higher levels of freedom within our physical, emotional, spiritual and psychological experience.
THANK YOU TO OUR SPONSORS
Title Sponsor:
Theater of the Mind is supported in part by Science Sandbox, an initiative of the Simons Foundation.
Premiere Sponsors:
Additional support provided by:
Dry Cleaning Sponsor:
