Mind Hacks™: Tips & Tools for Using Your Brain (13 page)

— William Bardel

A powerful illusion of brightness shows how our brain takes scene structure and
implied lighting into account when calculating the shade of things.

Take a look at Adelson’s checker shadow illusion in
Figure 2-19
.

Figure 2-19. Adelson’s checker shadow — which is brighter, A or B?

We would all agree that one sees a checkerboard with a pillar standing in one corner.
Illumination obviously comes from the top-right corner, as the shadow on the checkerboard
tells us immediately (and we know how important shadows are for informing what we see
[
Fool Yourself into Seeing 3D
]
).
All of this is perceived at one rapid glance, much faster than this sentence can be read
(lest written!).

Now let’s ask the following question: which square is brighter, A or B? The obvious
answer is B, and I agree. But now change the context by looking at
Figure 2-20
. The unmasked grays are from the
two squares A and B, and unquestioningly the two shades of gray are identical (in fact,
the entire figure was constructed just so).

You can prove it to yourself by cutting out a mask with two checker square–size holes
in it, one for A and one for B, and putting it over the original checkerboard (
Figure 2-19
).

Figure 2-20. This checkerboard is the same as the first, except for the added bars — now does A
look brighter than B?

If squares A and B in the first case have clearly differing brightness and in
the second case they have the same, what gives? Surely the two alternatives exclude each
other? The solution in a nutshell: brightness depends on context.

There is a good reason that visual scientists describe their experiments using the
term
luminance
rather than brightness. Luminance is a physical
measure, effectively counting the number of light quanta coming from a surface, then
weighting them by wavelength with regard to their visibility. (The unit of measurement, by
the way, is candela per square meter, cd/m
²
. A candela was
originally defined as the light from a standard candle 1 foot away.)

Brightness, on the other hand, is a subjective measure — something your brain constructs
for your conscious experience. It depends on previous history (light adaptation), the
immediate surroundings (contrast effects), and context (as here). It has no dimension but
can be measured using psychophysical techniques.

What exactly is happening when comparing
Figure 2-19
and
Figure 2-20
? Well, when I initially asked,
“Which square is brighter?”, I knew you would give the deeper answer, namely the lightness
quality of the substance the squares are made of. I knew you — or your smart visual
system — would assess the scene, interpret it as a 3D scene, guess the shadowed and lit
parts, predict an invisible light source, measure incoming light from the squares,
subtract the estimated effect of light versus shadow, and give a good guess at the true
lightness — the lightness that we would expect the checker squares to really have given the
way they appear in the scene they’re in. With the mask applied (
Figure 2-20
), however, we create a very
different context in which a 3D interpretation does not apply. Now the two squares are not
assumed to be lit differently, no correction for light and shadow needs to be applied, and
the brightness becomes equal. The luminance of squares A and B is always identical, but
due to different context, the perceived brightness changes.

By the way: there are more places in that figure where luminances are equal, but
brightness differs, and hunting for those is left as an exercise for the gentle
reader.

This striking checker shadow illusion by Ted Adelson teaches us quite a number of
things: it demonstrates how much unconscious scene computation goes on in our visual brain
when it applies inverse perspective and inverse lighting models. It shows us how strongly
luminance and brightness can differ, giving rise to perceptual constancies, here light
constancy. It also demonstrates the “unfairness” of the term “optical illusion”: the first
answer you gave was not wrong at all; in fact, it was the answer one would be interested
in, most of the time. Imagine the checkerboard were like a puzzle, with missing pieces,
and you had to hunt for a matching piece. Material property is what we need then,
independent of lighting. In fact, estimating the “true” material properties independent of
context is a very hard computational problem and one that hasn’t been solved to a
satisfying degree by computer vision systems.

Correction of surface perception for light and shadow conditions is such a
basic mechanism of our perception — and one that normally operates nearly perfectly — that
very artificial situations must be created by the accompanying figures for it to reveal
itself. That is why we need technical help taking photographs: since photos are normally
viewed under different lighting conditions compared to the original scene, professional
photographers need to go a long way arranging lighting conditions so that the impression
at viewing is the one that is desired.

— Michael Bach

We can use a little-known illusion called the Pulfrich Effect to hack the brain’s
computation of motion, depth, and brightness — all it takes is a pair of shades and a
pendulum.

Make a pendulum out of a piece of string and something heavy to use as a weight, like
a bunch of keys. You’ll also need a pair of sunglasses or any shaded material.

Ask a friend to swing the pendulum in front of you in a perpendicular plane, and make
sure it’s going exactly in a straight line, left to right. Now, cover one of your eyes
with the shades (this is easiest if you have old shades and can poke one of the lenses
out). Keep both eyes open! You’ll see that the pendulum now seems to be swinging back and
forth as well as side to side, so that it appears to move in an ellipse. The two of you
will look something like
Figure 2-21
.

Figure 2-21. Matt and Tom use sunglasses and a pendulum made out of a bootlace to test the
Pulfrich Effect

Show your friend swinging the pendulum how you see the ellipse, and ask her to
swing the pendulum in the opposite manner to counteract the illusion. Now the pendulum
appears to swing in a straight line, and the thing that seems odd is not the distance from
you, but the velocity of the pendulum. Because it really is swinging in an elliptical
pattern, it covers perceived distance at an inconsistent rate. This makes it seem as if
the pendulum is making weird accelerations and decelerations.

The classic explanation for the Pulfrich is this: the shading slows down the
processing of the image of the object in one eye (lower brightness means the neurons are
less stimulated and pass on the signal at a slower rate
[
Why People Don’t Work Like Elevator Buttons
]
); in effect, the
image reaches one eye at a delay compared to when it reaches the other eye. Because the
object is moving, this means the position of the image on the retina is slightly shifted.
The difference in image perception between the two retinas is used by the visual system to
compute depth
[
Depth Matters
]
. The
slight displacement of the image on the retina of the shaded eye is interpreted as an
indication of depth, as in
Figure 2-22
.

Figure 2-22. The geometry of the Pulfrich Effect: although the pendulum is, in reality, at point
1, the delay in processing makes it appear to be at point 2 to the shaded eye. When the
eyes are taken together, the pendulum therefore appears to be at point 3, at a different
depth.

This explanation puts the confounding of depth and motion on the geometry of the
situation — the point of confusion lies in the world, not in the brain.

Taking recordings of the responses of individual brain cells, Akiyuki Anzai and
colleagues have shown that this isn’t the whole story. The confounding of motion and depth
goes deeper than a mathematical ambiguity that arises from computing real-world
interpretations from the visual images on the retinas.

It seems that most of the neurons in the primary visual cortex are sensitive
to motion and depth in combination. These neurons are optimally responsive to some
combination of motion and depth; what makes up that optimum combination can be varying
amounts of motion and depth. This means when you see something and judge its distance your
brain always also makes a judgment about its velocity, and vice versa. From the first
point in your primary visual cortex where information from the two eyes is combined (i.e.,
very early in visual processing), motion and depth are coupled. You don’t get a sense of
one without getting a sense of the other.

This may result from the use of motion parallax to detect depth
[
Depth Matters
]
. Moving your head is one of
the basic ways of telling how far away something is (you can see spitting cobras using
motion parallax by shifting their heads from side to side to work out how far to spit). It
works even if you have the use only of one eye.

The joint encoding theory explains why you can get Pulfrich-like effects in situations
with less obvious geometry. If you watch television snow with one eye shaded, you will see
two sheets of dots, one in front of the other and one moving to the left and one moving to
the right. The reasons for this are complex but rest on the way our eyes try and match
dots in the images for both eyes and use this matching to calculate depth (stereoscopic
vision). Adding a shade to the image in one eye creates a bias so that instead of
perceiving all the dots at a single average depth we see two sets of skewed averages, and
because depth and motion are jointly encoded, these two planes move as well (in opposite
directions).

The Pulfrich Effect can be used to create 3D effects for television, as long as people
are willing to watch with one eye shaded. It’s hard to do since the motion of the
image/camera has to be smooth to create a consistent illusion of depth, but it has been done.
¹