You See With Your Hands

What's going on here

Two things, and the second one is the exhibit.

With the trail off you have a coin-sized disc and no picture. You can move it anywhere you like and still never have the thing in front of you -- because the moment the disc leaves a place, that place is gone. Plenty of information is arriving. None of it is accumulating.

Turn the trail on, sweep the same hole over the same frame, and a whole scene appears. Nothing has changed about what you are being shown. The aperture is the same size, the content is the same, your movements are the same. The only difference is whether anything is kept -- and that single difference is the difference between a frustrating little spotlight and the feeling of looking at a picture.

Now the uncomfortable part

You are the second version. Right now.

The high-resolution part of your retina -- the fovea -- covers about two degrees of your visual field. That is roughly a thumbnail at arm's length. Everything outside it is smeared, poorly coloured, and much worse than you believe it to be. To compensate, your eyes jump about three or four times every second, in flicks called saccades, of which you make something like two hundred thousand a day. You are almost never told about any of them.

So the rich, stable, wall-to-wall visual world you are certain you are sitting in is assembled exactly the way that scene is: a small sharp hole, moved constantly, with something keeping the results. You have never once seen a whole room. You have swept one.

This is the observation the museum has been circling since Exhibit № 1, where your brain papers over a hole the size of a grapefruit, and Exhibit № 6, where enormous things change in front of you and you miss them. Both of those are much less surprising once you accept that the completeness was never there to begin with.

This isn't a metaphor -- it's a method

The frame above is a real laboratory technique with a real name. In 1975 George McConkie and Keith Rayner built the moving window: they tracked a reader's eyes and, live, replaced every letter outside a small window around the fixation point with junk. Readers sailed on. Reading speed barely suffered until the window was squeezed down to something like fifteen characters -- and a great many readers never noticed that the entire rest of the page was garbage the whole time.

That is precisely what the frame above has you doing with a pointer. They had simply automated the hole and attached it to the eye, which is where yours has been attached your whole life.

The trick is older still. In 1862 Zöllner described what happens when a drawing is slid along behind a narrow slit: observers report seeing the whole shape, cleanly, despite never having more than a sliver of it on the retina at any instant. Helmholtz worried at it, and a century later people were still arguing about it under the splendid name anorthoscopic perception. A picture assembled from parts that were never simultaneously present is not an exotic laboratory curiosity. It is the ordinary case. It just took a slit to make it visible.

Two experiments that do the heavy lifting

Saying perception is active is easy. These are the two studies that make it hard to wriggle out of.

The kitten carousel (Held and Hein, 1963). Two kittens, raised in the dark, are placed in a carousel. One walks. The other rides in a gondola mechanically yoked to the first, so it is carried through an almost identical path and receives an almost identical stream of images. Same input. Only one of them is producing it. Afterwards the active kitten behaves normally -- puts a paw out to meet a surface, blinks at a looming object, declines to walk off a visual cliff. The passive one, after the same hours of the same pictures, does not. It had been shown everything and had learned to see almost nothing. (Let loose to move on its own afterwards, it caught up within days.)

Sensory substitution (Paul Bach-y-Rita, 1969). A camera's image is converted into a grid of vibrating points pressed against a blind person's back. With practice, users stopped reporting tingles on their skin and began reporting objects, out in the room, at a distance -- one famously flinched when the camera was zoomed without warning. But that externalising only happened when the person aimed the camera themselves. When an experimenter did the aiming, the very same signals stayed what they physically were: patterns on a back. The entire difference between having a sensation and having a world was who was doing the moving.

The theory this room is named for

In 1991 Francisco Varela, Evan Thompson and Eleanor Rosch published The Embodied Mind and proposed something that still hasn't been digested: that perceiving is not receiving. On their account -- enactivism -- perception is a skill you exercise, not a picture you are handed. The world doesn't arrive. You go and get it, continuously, with a body that moves, and what you experience is the shape of that activity rather than the contents of an inner screen.

They were building on J. J. Gibson, whose ecological approach had already insisted through the 1960s and 70s that an observer is not a camera on a tripod: it moves, and the moving is how the information gets picked up in the first place. Kevin O'Regan and Alva Noë then gave the idea its sharpest modern statement in 2001, arguing that seeing consists in mastery of the rules linking how you move to how the input changes -- that knowing what would happen to this scene if you turned your head is not a fact about your vision, it is what your vision is. Noë later put it as bluntly as it can be put: seeing is more like touching than like photography. You don't have a photograph of the room. You have the practised ability to find out about any part of it whenever you want, so fast and so reliably that it feels like already having it. The completeness is a line of credit, not a warehouse.

And there's a question this tradition likes, which is worth sitting with. It is Merleau-Ponty's, from 1945, and Michael Polanyi worried at the same one. Someone who has used a white cane for years does not feel vibration in their fingers. They feel the kerb -- out there, at the tip. The cane has gone transparent, the way your own hand is transparent to you. So: where does that person's perceiving happen? At the hand? At the tip? Somewhere along the stick? If your answer is "in the brain, obviously," you now have to explain why it doesn't feel like that, ever, to anyone -- and why Bach-y-Rita's volunteers only got the far-away version when they were the ones moving.

How much of this to believe

The moderate version is close to consensus: perception is active, it takes time, it depends on what your body can do, and you have far less detail in hand at any instant than you think. Nobody serious disputes this any more.

The strong version -- that there are no internal representations at all, that the brain is not building any model and the "picture" talk should be thrown out entirely -- is a genuine fight. Most cognitive scientists think the brain demonstrably does construct models, and that enactivism overcorrects. The enactivists reply that "model" is doing suspiciously heavy lifting in that sentence. Nobody has settled it, which is why the needle sits where it does.

Two useful dissenters. Daniel Dennett got to the emptiness first, in 1991, and drew a different moral: the brain doesn't laboriously fill in the missing detail, it simply never bothers to represent its absence, and there is no inner screen for the filling-in to happen on. He agrees the richness is a judgement rather than a picture -- he just thinks you don't need a whole new philosophy of mind to say so. Meanwhile the predictive-processing people -- Andy Clark, Anil Seth -- agree wholeheartedly that perception is active and then part company hard: for them the brain is running a rich internal model and perception is the model being corrected by the world, which is close to the exact thing enactivism was trying to get rid of. Two camps, same starting observation, opposite conclusions. That is roughly what an unsolved problem looks like from inside.

One last thing, in the museum's favourite register. Varela spent his later years insisting that studying consciousness required taking first-person experience seriously as data, not politely setting it aside until the measurements improved. Whatever survives of enactivism, that part has aged extremely well. His timescales are also quietly running inside the machine in Exhibit № 38, which is a nice thing for a philosopher to be able to say.

Take it further

The Embodied Mind (Varela, Thompson & Rosch, 1991) is the founding text and is still startling; Alva Noë's Action in Perception (2004) is the sharpest short statement of the seeing-is-touching idea, and O'Regan & Noë's 2001 paper in Behavioral and Brain Sciences is the formal one, published with several dozen pages of other researchers arguing with it, which is the best way to read anything. The general area lives under enactivism, and the specific claim that your visual world is far emptier than it feels has the excellent name the grand illusion.

For the experiments: the sensory substitution story runs from Bach-y-Rita's 1969 Nature paper to the tongue-display devices people use today, and it is one of the strangest and most hopeful corners of neuroscience. Held & Hein's kittens are in almost every perception textbook. And if you'd like to feel the moving window rather than read about it, Exhibit № 6 is the same family of trick, played for laughs.