Visual perception: depth cues, object recognition, top-down vs. bottom-up processing

Intuition Beginner

Seeing feels effortless. You open your eyes and the world simply appears: shapes, colours, distance, motion. But this ease hides enormous computation. Your brain is not passively receiving a picture from the world. It is actively building one. The light hitting each eye is a flat, two-dimensional pattern, yet you experience a rich three-dimensional scene. Understanding how the brain turns flat retinal images into depth, objects, and meaning is the project of visual perception.

Each eye receives only a flat image, yet you perceive depth. Binocular depth cues come from comparing the slightly different images from your two eyes — your eyes sit a few centimetres apart, so each sees the world from a slightly different angle. The brain fuses those two views and extracts distance from their difference. Monocular cues work with one eye closed. Relative size, overlap (one object partly hiding another), linear perspective (railroad tracks seeming to converge), texture gradients (brickwork growing finer with distance), and motion parallax all signal depth through a single eye.

Recognising an object is harder than it feels. The same cup casts countless different images on your retina as it rotates, moves, and is lit differently — yet you know it is the same cup. The brain solves this by combining two strategies. Bottom-up processing builds perception up from raw features: edges, colours, motion. Top-down processing uses expectations, memories, and context to shape what you see. A blurry shape in a kitchen is read as a toaster far faster than the same shape floating in empty space.

Visual illusions are not glitches. They reveal the shortcuts and assumptions your brain relies on every moment. The Mueller-Lyer arrows, the Ames room, impossible figures — each works because your visual system applies a rule that usually helps (rectangles, straight lines, light from above) to a scene engineered to break it. Studying illusions is studying normal perception at the points where its assumptions fail. What you see is always the brain's best guess, not a faithful copy of the world.

Visual Beginner

The diagram traces the path from light entering the eye to your conscious experience of a scene. Light strikes the retina, travels through the optic nerve to the thalamus, and reaches the primary visual cortex at the back of the brain. From there it splits: the ventral stream runs forward into the temporal lobe to identify objects (the "what" path), while the dorsal stream runs into the parietal lobe to track location and guide action (the "where/how" path). Depth cues, object recognition, and top-down expectations all act along these routes.

Worked example Beginner

Imagine a friend throws you a ball. As it nears, your two eyes receive slightly different images — binocular disparity tells your brain how far away it is. The ball's image grows rapidly on your retina; that rate of expansion (a monocular cue) signals it is approaching fast. Your dorsal stream computes the ball's path and guides your hand to the right spot in real time, too fast for conscious thought. Meanwhile your ventral stream has already identified it as a tennis ball, not an apple, drawing on colour, texture, and shape. Catching is perception and action fused, performed at a speed that hides the machinery doing the work.

Check your understanding Beginner

Formal definition Intermediate+

Retinal processing

Vision begins at the retina, a thin neural sheet lining the back of the eye. Photoreceptors transduce light: rods handle low-light (scotopic) vision and are colour-blind but extremely sensitive; cones handle daylight (photopic) vision and colour, in three classes tuned to short (S, ~430 nm), medium (M, ~530 nm), and long (L, ~560 nm) wavelengths. Photoreceptors synapse onto bipolar cells, which in turn drive ganglion cells whose axons form the optic nerve. Horizontal and amacrine cells mediate lateral interactions.

The foundational concept for retinal (and cortical) processing is the receptive field: the region of visual space and the stimulus pattern that changes a given neuron's firing. Ganglion-cell receptive fields are centre-surround antagonistic — a spot of light in the centre excites an ON-centre cell while light in the surround inhibits it (and vice versa for OFF-centre cells). This lateral inhibition sharpens contrast at edges: the cell responds most strongly to local brightness differences, not to uniform illumination. The retina does not transmit a faithful luminance map; it transmits a contrast-enhanced signal.

From retina to V1

Ganglion-cell axons travel via the optic nerve, partially cross at the optic chiasm (so each hemisphere receives the contralateral visual field), and relay through the lateral geniculate nucleus (LGN) of the thalamus. The LGN preserves a retinotopic map: neighbouring retinal locations drive neighbouring LGN neurons. From the LGN, fibres project to the primary visual cortex (V1, striate cortex) in the occipital lobe, where the mapping remains retinotopic, with disproportionate cortical area devoted to the fovea (the cortical magnification factor).

Cortical visual processing

V1 is where feature analysis begins in earnest. Hubel and Wiesel distinguished simple cells (orientation-tuned, with segregated ON and OFF subfields responding to edges at a particular angle and position) and complex cells (orientation-tuned but less position-specific, often direction-selective). V1 neurons are tuned for orientation, spatial frequency, and binocular disparity. Beyond V1, processing fans into extrastriate areas with increasing specialisation: V2 continues contour and border-ownership analysis; V4 is associated with colour and shape; V5/MT is devoted to motion. This functional segregation underwrites the two-stream architecture treated in the next section.

Depth perception

The brain fuses multiple cues to recover the third dimension from a two-dimensional retinal projection.

Binocular cues.

• Binocular disparity (stereopsis): the two eyes' images differ because the eyes are separated by roughly 6.3 cm. Disparity-sensitive neurons in V1 and beyond encode the horizontal offset between corresponding points. For a fixation distance $z$ and interocular separation $a$, the disparity for a point at depth $z+\Delta z$ is approximately

$$\eta \approx \frac{a\Delta z}{z^2}$$

in angular units (for $\Delta z\ll z$). Near fixation, disparity is large and depth resolution is fine; it degrades rapidly with distance.

• Convergence: the inward rotation of the eyes to fixate a near object; the extraocular muscle signal provides a depth cue, useful only for nearby objects.

Monocular cues (pictorial and dynamic; available to one eye).

• Relative size: known-size objects projecting smaller retinal images are judged farther.
• Interposition (occlusion): a blocking object is nearer.
• Linear perspective: parallel lines converge with distance.
• Texture gradient: regular texture grows finer with distance.
• Relative height: objects higher in the visual field are usually farther (on the ground plane).
• Aerial perspective: atmospheric scattering makes distant surfaces hazier and bluer.
• Motion parallax: during observer motion, near objects drift faster across the retina than far ones.
• Accommodation: lens-shape change for near focus supplies proprioceptive feedback.

Gestalt principles of perceptual organisation

The Gestalt psychologists (Wertheimer, Koffka, Kohler) argued that the whole is not reducible to its parts: perception organises the sensory array into coherent figures according to grouping laws. These principles are now read as heuristics tuned to environmental statistics.

• Proximity — nearby elements group together.
• Similarity — elements sharing colour, size, orientation, or shape group together.
• Closure — gaps are filled to perceive complete figures.
• Good continuation (continuity) — lines are followed along their smoothest path.
• Common fate — co-moving elements group together.
• Figure-ground organisation — the field parses into a shaped figure against a shapeless ground, and ambiguous stimuli (Rubin's face-vase) admit reversals.
• Symmetry and common region — further grouping heuristics.

Object recognition theories

How does the brain identify objects across wildly varying viewpoints, lighting, and occlusion? Three families of theory dominate the intermediate literature.

• Feature analysis. Objects are represented as combinations of elementary features (edges, angles, colour patches). Recognition proceeds by detecting features and matching their configuration to stored descriptions. Hubel and Wiesel's hierarchical feature detectors gave this account a neural starting point.
• Recognition-by-components (Biederman, 1987). A small vocabulary of simple 3D shapes called geons (cylinders, bricks, wedges, etc., ~36 in number) combine like alphabet letters to compose objects. An object is recognised when its geon arrangement matches a stored structural description. The theory explains viewpoint-invariance: geons are recovered from nonaccidental properties of edges (collinearity, symmetry, parallelism, cotermination) that survive wide changes in viewpoint.
• Template matching. Stored templates are compared to the input, typically after normalising for size, position, and orientation. Pure template matching is brittle but reappears in modern form as feature-vector matching in computational models.

Perceptual constancies

The retinal image changes constantly, yet perceived object properties stay stable.

• Size constancy: perceived size stays constant across distance because the brain scales retinal size by perceived distance (Emmert's Law: perceived size $\propto$ retinal size $\times$ perceived distance).
• Shape constancy: a door is seen as rectangular whether open or closed.
• Colour constancy: surface colour stays stable across illuminants (a red apple looks red under sun, fluorescent, or tungsten light), achieved by estimating and discounting the illuminant.
• Lightness constancy: perceived reflectance is stable across illumination level.

Top-down processing and context

Perception is not driven solely by the sensory input (bottom-up). It is shaped by expectation, context, memory, and task — top-down influences. The word superiority effect (Reicher, 1969) shows that a letter is recognised more accurately when it appears in a real word than when it appears alone or in a nonword, demonstrating that higher-level lexical context feeds back to aid lower-level letter recognition. Contextual modulation in cortex and behavioural priming both reflect the same idea: what you already know about the scene changes what you see. Bottom-up and top-down processing run in parallel; the resulting percept is a weighted blend of sensory evidence and prior expectation.

Key model: the two cortical visual streams Intermediate+

Of the models that organise visual perception, the most consequential is the two-stream hypothesis: that cortical visual processing splits into a ventral "what" pathway and a dorsal "where/how" pathway. It reframes a pile of disparate findings (face areas, motion cortex, spatial neglect, optic ataxia, object agnosia) into a single architectural picture, and it ties together every theme in this unit — object recognition, depth, and the bottom-up/top-down interplay.

The original proposal: what vs. where

Ungerleider and Mishkin (1982) proposed the division on the basis of lesion studies in macaques. Monkeys with inferior temporal (ventral) lesions lost object discrimination (they could no longer tell a triangle from a square by shape), but their spatial localisation stayed intact. Monkeys with posterior parietal (dorsal) lesions showed the reverse pattern: they localised objects poorly but recognised them fine. The two deficits formed a double dissociation, the gold-standard evidence that two brain regions support two distinct functions. The ventral stream, running from V1 into the inferior temporal cortex, computes object identity; the dorsal stream, running from V1 into the posterior parietal cortex, computes spatial location.

The Milner-Goodale revision: what vs. how

A decade later, Milner and Goodale (1995) sharpened the theory by redefining the dorsal stream's job. Its function is not merely "where" (a spatial map) but "how" — the visual control of action. The evidence came largely from patient D.F., who suffered carbon-monoxide-induced occipital damage leaving her unable to recognise objects or match their orientation (a ventral-stream failure), yet able to reach out and grasp them with perfectly calibrated hand posture and grip size (an intact dorsal stream). D.F. could not describe a slot's orientation, but she could post a card through it. Her ventral stream was destroyed; her dorsal stream guided action without conscious perception.

The complementary dissociation appears in optic ataxia: patients with parietal (dorsal) damage can describe an object's size and orientation but misreach for it, fumbling the grasp. Ventral stream supports perceptual awareness and recognition; dorsal stream supports real-time visuomotor transformation, operating faster and more unconsciously.

Why this model organises the unit

The two-stream model earns its central place because it maps directly onto the unit's three themes.

Object recognition lives in the ventral stream. The progression from V1 simple cells (oriented edges) through V4 (shape, colour) to the fusiform face area and other category-selective regions is the biological implementation of bottom-up feature combination, refined by top-down expectation. The ventral stream is where geons, faces, and the word superiority effect play out.

Depth and spatial layout live in the dorsal stream. Motion parallax, optic flow, and the real-time use of binocular disparity for reaching are dorsal computations. The dorsal stream consumes the depth cues enumerated above and converts them into a coordinate frame centred on the effector (eye, hand, mouth), not on the world.

Bottom-up and top-down processing are stream-wide, not confined to one stream. Both pathways receive massive feedback from frontal and parietal regions. Top-down attention modulates firing in V4 and MT; expectation sharpens ventral-stream object responses; task demands retune dorsal visuomotor fields. The two-stream architecture is not a pair of feedforward pipes — it is two recurrent hierarchies, each integrating sensory evidence with prior knowledge.

Limits of the model

The model is an organising approximation, not a final description. The streams are not strictly segregated: they interact at multiple levels, share inputs, and converge in the frontal and parietal lobes. Both streams carry some spatial and identity information. The model also simplifies away a rich hierarchy (V2, V3, V4, LO, FFA, PPA, EBA on the ventral side; V3A, MT, MST, VIP, LIP, AIP on the dorsal side). And it says little about subcortical routes (the superior colliculus pathway to the amygdala, which supports blindsight). The model's value is heuristic: it gives the right first-pass map of visual cortex, and every refinement is defined relative to it.

Exercises Intermediate+

Advanced topics Master

Marr's three levels and his vision framework

David Marr (1982) argued that any information-processing system — including the visual system — must be understood at three levels.

1. Computational level. What is the system computing and why? For vision: recovering the 3D layout, surface properties, and identities of objects from a 2D intensity array. The problem is underdetermined without assumptions (the world is rigid, surfaces are continuous, light comes from above).
2. Algorithmic level. How is the computation carried out — what representation does it use and what algorithm operates on that representation?
3. Implementational level. How is the algorithm physically realised — in neurons, circuits, synapses?

Marr's warning was that confounding the levels produces sterile debates. Arguing about whether a neuron "really" computes an edge (implementational) without specifying the algorithm or the goal (computational) misses what the system is for.

Marr then proposed a staged framework for vision. From the intensity image the system derives the primal sketch — a description of edge segments, bars, blobs, and terminations at multiple scales. The primal sketch feeds the 2.5-D sketch, a viewer-centred representation of surfaces, their orientations, depths, and discontinuities, assembled from shading, texture, stereo, and motion cues. Finally, object recognition requires a 3-D model representation: an object-centred description that is stable across viewpoint. Marr's framework set the agenda for computational vision for forty years, and its computational-algorithmic-implementational trichotomy has become standard across cognitive science.

Bayesian models of perception

Modern computational perception is overwhelmingly Bayesian. The brain is modelled as maintaining a prior $P(\text{world})$ over possible scene states and a likelihood $P(\text{sensory data}\mid \text{world})$ describing how each state would generate the incoming signal. Perception is then the posterior:

$$P(\text{world}\mid \text{sensory data})\propto P(\text{sensory data}\mid \text{world})P(\text{world})$$

The Bayesian framework explains constancies, illusions, and individual differences uniformly. Size constancy is Bayesian discounting of distance: the brain combines a noisy retinal-size measurement with a prior over plausible object sizes, yielding a stable estimate. The Mueller-Lyer illusion is read as a rational inference given a prior that arrow-fin configurations usually signal 3D corners; the illusion is the optimal estimate under that prior, even if the stimulus is a flat drawing. Cross-cultural variation (Segall, Campbell, & Herskovits, 1966) becomes a difference in priors: carpentered-world inhabitants have stronger priors for right-angled structure, which inflates the illusion.

Predictive coding and free energy

Predictive coding (Rao & Ballard, 1999; Friston, 2005; Clark, 2013) gives the Bayesian story a neural mechanism. At each level of a cortical hierarchy, higher levels send predictions downward about the incoming signal, and lower levels send only the prediction error upward. The system continually updates its generative model to minimise prediction error. Attention is modelled as precision weighting: the brain increases the gain on prediction errors from reliable, task-relevant channels, making them more influential in revising the model.

Friston's free-energy principle generalises this further: any self-organising system that maintains itself in a stable set of states must, in effect, minimise variational free energy — an upper bound on surprise (negative log model evidence). On this view, perception and action are two faces of one imperative: the agent acts to bring sensory data into line with predictions, and perceives to bring predictions into line with data. Whether predictive coding is the literal cortical algorithm or an elegant computational metaphor is an open empirical question, but it is currently the dominant unifying framework in theoretical neuroscience.

Visual attention: feature integration and guided search

Visual attention selects some information for privileged processing from the overwhelming sensory array. Anne Treisman's feature integration theory (FIT) (Treisman & Gelade, 1980) distinguished two stages. A fast, parallel preattentive stage registers elementary features (colour, orientation, size, motion) across the whole field, producing a set of feature maps. A slow, serial attentional stage then binds features into coherent objects by focusing a "spotlight" on one location at a time. Searching for a green X among red Xs (a single-feature "pop-out" search) is fast and independent of set size; searching for a green X among green Os and red Xs (a conjunction requiring binding) is slow and roughly linear in the number of distractors. Wolfe's guided search refined FIT by allowing preattentive features to guide the attentional spotlight to likely targets, recovering much of the speed that pure serial search would forfeit.

A related debate concerns whether attention selects objects or regions of space. Evidence from object-based attention (Duncan, 1984; Egly, Driver, & Rafal, 1994) shows that attending to one feature of an object facilitates processing of its other features even at an unattended location — attention sticks to objects, not merely to coordinates.

Face perception

Faces are recognised with remarkable speed and expertise, and the brain devotes specialised machinery to them. The fusiform face area (FFA) (Kanwisher, McDermott, & Chun, 1997), on the fusiform gyrus, responds far more strongly to faces than to objects, and damage here produces prosopagnosia (face blindness) — an inability to recognise familiar faces despite intact early vision and object recognition. The FFA's specificity is contested (is it a face module or a generic expertise area for any over-trained category?), but the behavioural signature of face expertise is robust:

• The face inversion effect: faces are disproportionately hard to recognise when upside down, far more than objects are — suggesting holistic, configural processing specific to upright faces.
• The other-race effect: faces of one's own racial group are recognised more accurately and with a stronger holistic signature than other-race faces, an effect of perceptual experience rather than physiognomy.

Face processing is the cleanest case of category-selective ventral-stream cortex, alongside the parahippocampal place area (PPA, scenes) and the extrastriate body area (EBA, bodies).

Visual agnosias

Failing to recognise objects despite intact basic vision localises the stages of the ventral hierarchy. Lissauer (1890) distinguished:

• Apperceptive agnosia: patients cannot form a coherent percept — they fail at copying or matching shapes, even though acuity and fields are intact. The deficit sits at the construction of the perceptual representation.
• Associative agnosia: patients copy and match shapes competently yet cannot recognise what they have drawn — the percept is formed but cannot be linked to meaning. The deficit sits at the connection between perception and semantics.

Visual agnosias are natural double dissociations that support the stage-wise architecture of object recognition.

Depth perception and binocular neurons

Gian Poggio and colleagues (Poggio & Fischer, 1977; Poggio & Talbot, 1981) recorded from awake, behaving macaques and found neurons in V1 and V2 tuned to binocular disparity. Near cells, far cells, and tuned-inhibitory cells responded selectively to crossed disparity, uncrossed disparity, or zero disparity. These disparity-tuned neurons are the neural substrate of stereopsis: the population code over disparity-tuned cells encodes depth. Their existence also confirms that stereopsis is computed early, in V1, by combining inputs from corresponding retinal locations in the two eyes — a result consistent with the predictive-coding and bottom-up accounts.

Computational models of object recognition

Object recognition has been modelled computationally for half a century, and modern models are both engineering tools and theories of the ventral stream.

• HMAX (Riesenhuber & Poggio, 1999). A hierarchical model in which simple-cell-like S units compute tuned weighted combinations of their inputs and complex-cell-like C units compute a max-like pooling that yields position and scale invariance. HMAX showed that a hierarchy of simple nonlinear operations can produce view-tolerant, selectivity-preserving object units — a biologically plausible skeleton of the ventral stream.
• Deep convolutional neural networks (CNNs). Modern CNNs (Krizhevsky et al., 2012, and successors) extend the same architectural insight — alternating convolution and pooling followed by classifier layers — to much greater depth. Cadieu et al. (2014) showed that the internal representations of deep CNNs predict inferior-temporal (IT) cortex responses at least as well as the IT population predicts itself, a striking quantitative fit that established CNNs as the leading current model of the ventral stream. CNNs also reproduce human behavioural signatures (clustering by category, tolerance to identity-preserving transformations, susceptibility to some of the same category-confusion structure).
• The fit is imperfect. CNNs are fooled by adversarial examples — imperceptible perturbations that flip their classification — which the human ventral stream is largely immune to. CNNs are also more brittle under distribution shift. The implication for perception theory is that the ventral stream achieves its robustness not merely by depth and convolution but by additional constraints (recurrence, feedback, ecological training, body-grounded objectives) that current models capture only partly.

Neural-network interpretability and perception

If CNNs model the ventral stream, understanding them bears on perception itself. Feature visualisation (Olah et al.) synthesises the input that most strongly drives a given unit, revealing what each neuron "wants" — edges in early layers, textures and object parts in mid layers, entire objects in late layers. The progression recapitulates the ventral-stream hierarchy empirically. The discovery of adversarial examples is more unsettling: it shows that the high-confidence classification of a perturbed image is not grounded in features a human would recognise. The implication is that CNN and human perception share an architecture but differ in their invariances — a warning against equating a model's accuracy with a model's perceptual competence.

Connections Master

• Sensation and perception 29.03.01. This unit extends the foundations of the prerequisite (sensation, perception as construction, psychophysics, signal detection) into the specific machinery of vision — the cortical streams, depth cues, object-recognition theories, and top-down modulation. Read 29.03.01 first; this unit supplies the visual detail.

• Brain regions and function 29.02.02 pending. Every structure named here — V1, V2, V4, V5/MT, the fusiform gyrus, the parietal lobe, the LGN — is treated anatomically in the brain-regions unit. The two units interlock: 29.02.02 supplies the geography, this unit supplies the function.

• Neuroscience: brain and behaviour 29.02.01. The receptive fields, lateral inhibition, and hierarchical feature detection treated here are grounded in the cellular neuroscience (action potentials, synapses, receptive-field physiology of Hubel and Wiesel) of 29.02.01.

• Learning and memory 29.04.01. Perceptual learning — the improvement in discrimination produced by practice — reshapes ventral-stream representations and is a form of implicit learning. The other-race effect and radiologists' expert perception are cases of perceptual learning shaping what is seen.

• Cognition and intelligence 29.05.01. Visual attention, feature integration theory, and the top-down modulation of perception overlap with the cognitive-psychology treatment of attention, working memory, and executive control.

• Other sensory systems 29.03.03 pending (successor, proposed). This unit completes the visual strand; the proposed successor extends the comparative treatment to auditory and somatosensory perception, completing the sensory overview.

• Computer science / machine learning. Deep CNNs as models of the ventral stream (Cadieu et al., 2014), adversarial examples, and feature visualisation place visual perception in dialogue with deep learning. The cross-link runs both ways: neuroscience inspired CNNs, and CNNs now serve as the leading computational theory of the ventral hierarchy.

• Art and aesthetics 20.04.01. Linear perspective, texture gradient, colour constancy, and the Gestalt grouping laws are the perceptual machinery that representational and abstract art exploit. Painters are applied perception researchers, whether or not they describe themselves that way.

Historical and philosophical context Master

Helmholtz and unconscious inference

The modern study of visual perception begins with Hermann von Helmholtz, whose Handbuch der physiologischen Optik (1867) introduced unconscious inference: the claim that perception is a rapid, automatic act of reasoning from ambiguous sensory data to the most likely state of the world. The retinal image underdetermines the scene — the same projection could arise from infinitely many 3D configurations — so the brain must infer the cause. Helmholtz's framing seeded both the constructivist tradition (Gregory, Rock) and, more recently, the Bayesian revival, which makes the inference literal: perception as probabilistic reasoning under uncertainty. The deep continuity is the idea that what you see is not what is on the retina but what the retina most likely came from.

The Gestalt revolt

In early-twentieth-century Germany, Wertheimer, Koffka, and Kohler rejected the atomistic claim that perception is built by combining sensory elements. Their demonstration of the phi phenomenon (apparent motion from a sequence of static flashes) showed an experienced whole — motion — that no element possessed. From this they derived the program of identifying the laws by which the brain organises the sensory field into wholes. Their grouping principles (proximity, similarity, closure, common fate, figure-ground) remain in the textbook canon today, though they are now read as ecological heuristics rather than as expressions of a single Gestalt principle of isomorphism. The Gestalt insistence on the primacy of organised wholes over elements remains a standing counterweight to purely bottom-up, feature-combinatorial accounts.

Hubel and Wiesel: the cortical microarchitecture of vision

David Hubel and Torsten Wiesel's recordings from cat and macaque visual cortex (1959 onward) gave perception its cellular foundation. Their discovery of orientation-selective simple and complex cells in V1, the columnar organisation of orientation preference, and the alternating ocular-dominance columns transformed vision from a black box into a circuit. Their demonstration that early visual deprivation (monocular deprivation during a critical period) permanently rewired ocular-dominance columns founded the study of critical-period plasticity, linking perception to development and to the neuroplasticity strand. The work earned them the 1981 Nobel Prize and established the hierarchical feature-detector picture that object-recognition theory and computational models (HMAX, CNNs) have built on ever since.

The two streams and the perception-action revolution

The Ungerleider-Mishkin (1982) what/where division was the first clean cortical architecture for vision. The Milner-Goodale (1995) what/how revision — built on patient D.F. and on the optic-ataxia complement — was more radical. It argued that the dorsal stream is not a spatial-perception system but a visuomotor one, and that visual awareness and visual action are biologically separable. This challenged a long philosophical tradition (rooted in Locke and in much of cognitive science) that treated perception as the input to action — something first done, then used. Milner and Goodale inverted the relationship: perception and action are parallel, differently tuned visual systems, each serving a different evolutionary purpose. The debate over whether awareness is genuinely ventral-only or whether the streams merely favour different contents remains live.

Marr and the computational turn

Marr's Vision (1982), published posthumously, reframed perception as a computational problem and laid down the three-level methodology that structured the field. Marr's insistence that the computational problem must be understood before the algorithm, and the algorithm before the neurons, was a corrective against a purely bottom-up neurophysiology that recorded cells without asking what they were for. His concrete framework (primal sketch, 2.5-D sketch, 3-D model) has been overtaken in detail by modern hierarchical and Bayesian models, but the methodology is permanent: perception is a problem of inference from ambiguous data, and any account that does not state the inference being made is incomplete.

The CNN revolution and its discontents

The success of deep CNNs on image classification (Krizhevsky, Sutskever, & Hinton, 2012) and the subsequent demonstration that CNN representations predict ventral-stream activity (Yamins et al., 2014; Cadieu et al., 2014) produced a striking alignment: the engineering system that works best is also the best current theory of the biological system. The alignment re-opened old questions. Are CNNs theories of perception or merely useful tools that happen to fit? Do adversarial examples reveal a fundamental difference between artificial and biological vision, or just an incompleteness of current training regimes? The current chapter is still being written, but the convergence has made computational perception a genuinely two-way street between neuroscience and machine learning for the first time since Marr.

Bibliography Master

1. Marr, D., Vision: A Computational Investigation into the Human Representation and Processing of Visual Information (W. H. Freeman, 1982). The computational-level framework (primal sketch, 2.5-D sketch, 3-D model) and the three-level methodology that structured four decades of vision science.

2. Hubel, D. H. and Wiesel, T. N., "Receptive Fields, Binocular Interaction and Functional Architecture in the Cat's Striate Cortex," Journal of Physiology 160 (1962), 106-154. The discovery of orientation-selective simple and complex cells and the columnar architecture of V1.

3. Ungerleider, L. G. and Mishkin, M., "Two Cortical Visual Systems," in Ingle, Goodale, and Mansfield (eds.), Analysis of Visual Behavior (MIT Press, 1982), 549-586. The original what/where proposal splitting ventral and dorsal streams.

4. Milner, A. D. and Goodale, M. A., The Visual Brain in Action (Oxford University Press, 1995). The what/how revision of the two-stream hypothesis, built on patient D.F. and optic ataxia.

5. Goodale, M. A., Milner, A. D., Jakobson, L. S., and Carey, D. P., "A Neurological Dissociation Between Perceiving Objects and Grasping Them," Nature 349 (1991), 154-156. The case of patient D.F.: intact dorsal visuomotor control with destroyed ventral perceptual recognition.

6. Poggio, G. F. and Fischer, B., "Binocular Interaction and Depth Sensitivity in Striate and Prestriate Cortex of Behaving Rhesus Monkey," Journal of Neurophysiology 40 (1977), 1392-1405. The discovery of disparity-tuned neurons underlying stereopsis.

7. Treisman, A. and Gelade, G., "A Feature-Integration Theory of Attention," Cognitive Psychology 12 (1980), 97-136. The preattentive/attentive two-stage model and the binding problem.

8. Biederman, I., "Recognition-by-Components: A Theory of Human Image Understanding," Psychological Review 94 (1987), 115-147. The geon theory of viewpoint-invariant object recognition.

9. Kanwisher, N., McDermott, J., and Chun, M. M., "The Fusiform Face Area: A Module in Human Extrastriate Cortex Specialized for Face Perception," Journal of Neuroscience 17 (1997), 4302-4311. The identification of the FFA and the case for category-selective cortex.

10. Reicher, G. M., "Perceptual Recognition as a Function of Meaningfulness of Stimulus Material," Journal of Experimental Psychology 81 (1969), 275-280. The word superiority effect and direct evidence for top-down lexical context in letter perception.

11. Riesenhuber, M. and Poggio, T., "Hierarchical Models of Object Recognition in Cortex," Nature Neuroscience 2 (1999), 1019-1025. The HMAX model of hierarchical feedforward object recognition.

12. Rao, R. P. N. and Ballard, D. H., "Predictive Coding in the Visual Cortex," Nature Neuroscience 2 (1999), 79-87. The predictive-coding model of cortical hierarchical inference via prediction error.

13. Friston, K., "A Theory of Cortical Responses," Philosophical Transactions of the Royal Society B 360 (2005), 815-836. The free-energy principle as a unifying account of perception and action.

14. Clark, A., "Whatever Next? Predictive Brains, Situated Agents, and the Future of Cognitive Science," Behavioral and Brain Sciences 36 (2013), 181-204. The predictive-processing synthesis and its philosophical implications.

15. Cadieu, C. F., Hong, H., Yamins, D. L. K., Pinto, N., Ardila, D., Solomon, E. A., Majaj, N. J., and DiCarlo, J. J., "Deep Neural Networks Rival the Representation of Primate IT Cortex for Core Visual Object Recognition," PLoS Computational Biology 10 (2014), e1003963. The quantitative demonstration that deep CNN representations predict IT cortex responses.

16. Krizhevsky, A., Sutskever, I., and Hinton, G. E., "ImageNet Classification with Deep Convolutional Neural Networks," in Advances in Neural Information Processing Systems 25 (2012), 1097-1105. The breakthrough that re-established deep CNNs and triggered the modern alignment between machine learning and vision science.

17. Yuille, A. L. and Kersten, D., "Vision as Bayesian Inference: Analysis by Synthesis?," Psychological Review 113 (2006), 764-784. The Bayesian unification of constancies, illusions, and perception-as-inference.

18. Segall, M. H., Campbell, D. T., and Herskovits, M. J., The Influence of Culture on Visual Perception (Bobbs-Merrill, 1966). The cross-cultural Mueller-Lyer studies that reframed illusions as products of learned environmental priors.

19. Gleitman, H., Gross, J., and Reisberg, D., Psychology, 8th ed. (W. W. Norton, 2011). Ch. 5 gives the intermediate-level treatment of perception, depth cues, Gestalt principles, and object recognition.

20. Myers, D. G. and DeWall, C. N., Psychology, 13th ed. (Worth, 2021). Ch. 6 provides the introductory survey of sensation and perception against which the beginner tier of this unit is calibrated.