The Zanvyl Krieger Mind/Brain Institute

Rudiger von der Heydt is a professor emeritus of neuroscience and a researcher in the Zanvyl Krieger Mind/Brain Institute. He received his BSc and MSc (Dipl. Phys.) from the University of Munich, Germany, and his PhD (Dr. sc. nat.) from the Swiss Federal Institute of Technology, Zurich, Switzerland. He joined the Hopkins Neuroscience Department in 1993. His research contributed to the understanding of the neural basis of visual perception. Most cited are his studies on illusory contours and figure-ground organization.

Neurophysiology of the Visual System

The visual sense, as much as the sense of touch, is a spatial sense. We see the world three-dimensional even though vision is based on just two-dimensional images, the optical projections on the retinae. In perception we can segregate figure and ground although objects often occlude one another so that foreground and background structures are cluttered up in the image. Perception decomposes a scene into objects and represents them in their proper dimensions in spite of varying image size and perspective distortion, and in their proper colors in spite of changing illumination. While the interpretation of images is technically very difficult, the visual system performs its task with great ease, so that we hardly become aware of it.

The investigation of the neural processes underlying visual perception is the theme of research of this laboratory. The methods are micro-electrode recording from cells of the visual cortex in awake behaving monkeys, psychophysical experiments in humans and monkeys, and mathematical modeling. We are currently working on two main projects. One relates to the mechanisms of 3D form perception from binocular and monocular cues, the other to the coding of features as parts of objects. We use subjective perceptual phenomena as tools in relating the neural signals to the hypothetical representations and processes underlying perception. For example, the changing depth of an ambiguous display, such as the Necker cube, can be used to probe for the neural mechanisms of depth perception, the tendency in figure/ground displays to perceive contours as part of the figure (rather than the ground) can be used to probe for mechanisms of object-related feature representation.

Example of Research

Border-ownership coding in the visual cortex. The bar graph on the right shows the responses (mean firing rate) of an area V2 neuron to the visual stimuli shown on the left. Ellipses mark the conventional receptive field of the neuron. In each case, the receptive field is stimulated with the same local edge, but in a different image context. Note that the neuron responds more strongly if that edge perceptually belongs to a figure on the lower left. We find border-ownership selectivity in about 50% of the cells in V2. Thus, although neurons of V2 have tiny receptive fields and signal local contour features (orientation, color gradient), they also code the global figure-ground relationship. The elusive ‘gestalt principles’ of perceptual organization have their correlates in the neural mechanisms of image segmentation.

PubMed Listings

• Analysis of spiking synchrony in visual cortex reveals distinct types of top-down modulation signals for spatial and object-based attention.

  Wagatsuma N, Hu B, von der Heydt R, Niebur E. PLoS Comput Biol. 2021 Mar 25;17(3):e1008829. doi: 10.1371/journal.pcbi.1008829. eCollection 2021 Mar. PMID: 33765007 Free PMC article.

• Searching for object pointers in the visual cortex.

  Zhu SD, Zhang LA, von der Heydt R. J Neurophysiol. 2020 May 1;123(5):1979-1994. doi: 10.1152/jn.00112.2020. Epub 2020 Apr 15. PMID: 32292110 Free PMC article.

• Figure-Ground Organization in Natural Scenes: Performance of a Recurrent Neural Model Compared with Neurons of Area V2.

  Hu B, von der Heydt R, Niebur E. eNeuro. 2019 Jun 28;6(3):ENEURO.0479-18.2019. doi: 10.1523/ENEURO.0479-18.2019. Print 2019 May/Jun. PMID: 31167850 Free PMC article.

• Figure and ground: how the visual cortex integrates local cues for global organization.

  von der Heydt R, Zhang NR. J Neurophysiol. 2018 Dec 1;120(6):3085-3098. doi: 10.1152/jn.00125.2018. Epub 2018 Jul 25. PMID: 30044171 Free PMC article.

• Figure-ground organization in the visual cortex: does meaning matter?

  Ko HK, von der Heydt R. J Neurophysiol. 2018 Jan 1;119(1):160-176. doi: 10.1152/jn.00131.2017. Epub 2017 Oct 4. PMID: 28978761 Free PMC article.

• Williford JR, von der Heydt R (2016) Figure-Ground Organization in Visual Cortex for Natural Scenes. eNeuro 3-6:1-15. http://eneuro.org/content/3/6/ENEURO.0127-16.2016
• Wagatsuma N, von der Heydt R, Niebur E (2016) Spike synchrony generated by modulatory common input through NMDA-type synapses. J Neurophysiol 116:1418.
• von der Heydt R (2015) Figure-ground organization and the emergence of proto-objects in the visual cortex. Frontiers in Psychology 6:1695. http://journal.frontiersin.org/article/10.3389/fpsyg.2015.01695/full
• Martin AB, von der Heydt R (2015) Spike synchrony reveals emergence of proto-objects in visual cortex. J Neurosci 35:6860-6870. http://www.jneurosci.org/content/35/17/6860.full.pdf+html
• Hu B, von der Heydt R, Niebur E (2015) A neural model for perceptual organization of 3D surfaces. pp 1-3. IEEE. http://ieeexplore.ieee.org/xpl/mostRecentIssue.jsp?punumber=7075844
• Russell AF, Mihalas S, von der Heydt R, Niebur E, Etienne-Cummings R (2014) A model of proto-object based saliency. Vision Res 94:1-15. https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3902215
• Martin AB, von der Heydt R (2015) Spike synchrony reveals emergence of proto-objects in visual cortex. J Neurosci 35:6860-6870.
• O'Herron P, von der Heydt R (2013) Remapping of border ownership in the visual cortex. The Journal of Neuroscience 33:1964-1974.
• von der Heydt R (2013) Neurophysiological constraints on models of illusory contours. Cognitive Neuroscience 4:49-50.
• von der Heydt R (2013) Image parsing mechanisms of the visual cortex. In: The New Visual Neurosciences (Werner JS, Chalupa LM, eds), pp 665-678. Cambridge, Massachusetts: Massachusetts Institute of Technology.
• Williford JR, von der Heydt R (2013) Border-ownership coding. Scholarpedia 8:30040.
• Ardila D, Mihalas S, von der Heydt R, Niebur E (2012) Medial axis generation in a model of perceptual organization. In: 46th Annual Conference on Information Sciences and Systems IEEE-CISS-2012 pp 1-4. IEEE Information Theory Society.
• Wagemans J, Elder JH, Kubovy M, Palmer SE, Peterson MA, Singh M, von der Heydt R (2012) A century of Gestalt psychology in visual perception: I. Perceptual grouping and figure-ground organization. Psychol Bull 138:1172-1217.
• Mihalas S, von der Heydt R, Niebur E (2011) A network model of multiplicative attentional modulation. In: 45th Annual Conference on Information Sciences and Systems IEEE-CISS-2011 pp 1-6. IEEE Information Theory Society.
• Mihalas S, Dong Y, von der Heydt R, Niebur E (2011) Mechanisms of perceptual organization provide auto-zoom and auto-localization for attention to objects.Proceedings of the National Academy of Sciences 108:7583-7588.
• Sugihara T, Qiu FT, von der Heydt R (2011) The speed of context integration in the visual cortex. J Neurophysiol 106:374-385.
• O'Herron PJ, von der Heydt R (2011) Representation of object continuity in the visual cortex. J Vision 11:12,1-12,9.
• von der Heydt R (2011) Contour-, Surface-, and Object-Related Coding in the Visual Cortex. In: Computer Vision: From Surfaces to 3D Objects (Tyler CW, ed), pp 145-162. Boca Raton: Chapman & Hall/CRC.
• Zhang NR, von der Heydt R (2010) Analysis of the context integration mechanisms underlying figure-ground organization in the visual cortex. J Neurosci 30:6482-6496.
• O'Herron PJ, von der Heydt R (2009) Short-Term Memory for Figure-Ground Organization in the Visual Cortex. Neuron 61: 801-809.
• Dong Y, Mihalas S, Qiu FT, von der Heydt R, Niebur E (2008) Synchrony and the binding problem in macaque visual cortex. J Vision 8: 1-16.
• Qiu FT, Sugihara T, von der Heydt R (2007) Figure-ground mechanisms provide structure for selective attention. Nat Neurosci 10: 1492-1499.
• Craft E, Schuetze H, Niebur E, von der Heydt R (2007) A neural model of figure-ground organization. J Neurophysiol 97: 4310-4326.
• Qiu FT, von der Heydt R (2007) Neural representation of transparent overlay. Nat Neurosci 10: 283-284.
• von der Heydt R, Pierson R (2006) Dissociation of color and figure-ground effects in the watercolor illusion. Spatial Vision 19: 323-340.
• Qiu, F.T., and von der Heydt, R. (2005). Figure and ground in the visual cortex: V2 combines stereoscopic cues with Gestalt rules. Neuron 47, 155-166.
• von der Heydt R, Macuda TJ, Qiu FT (2005) Border-ownership dependent tilt aftereffect. J Opt Soc Am (A) 22: 2222-2229.
• Friedman HS, Zhou H, von der Heydt R (2003) The coding of uniform color figures in monkey visual cortex. J Physiol (Lond) 548: 593-613.
• von der Heydt R (2003) Image parsing mechanisms of the visual cortex. In: The Visual Neurosciences (Werner JS, Chalupa LM, eds), pp 1139-1150. Cambridge, Mass.: MIT press.
• von der Heydt R, Friedman HS, Zhou H (2003) Searching for the neural mechanisms of color filling-in. In: Filling-in: From Perceptual Completion to Cortical Reorganization (Pessoa L, De Weerd P, eds), pp 106-127. Oxford: Oxford University Press.
• von der Heydt R, Zhou H, Friedman HS (2003) Neural coding of border ownership: Implications for the theory of figure-ground perception. In: Perceptual Organization in Vision: Behavioral and Neural Perspectives (Behrmann M, Kimchi R, Olson CR, eds), pp 281-304. Mahwah: Lawrence Erlbaum Associates.
• von der Heydt R, Zhou H, Friedman HS (2000) Representation of stereoscopic edges in monkey visual cortex. Vision Research 40: 1955-1967.
• Zhou H, Friedman HS, von der Heydt R (2000) Coding of border ownership in monkey visual cortex. J Neuroscience 20: 6594-6611.
• Friedman HS, Zhou H, von der Heydt R (1999) Color filling-in under steady fixation: Behavioral demonstration in monkeys and humans. Perception 28: 1383-1395.
• Heitger F, von der Heydt R, Peterhans E, Rosenthaler L, Kübler O (1998) Simulation of neural contour mechanisms: representing anomalous contours. Image and Vision Computing 16: 409-423.
• von der Heydt R (1994) Form analysis in visual cortex. In: The Cognitive Neurosciences (Gazzaniga MS, ed), pp 365-382. Cambridge, Mass.: MIT Press.
• Heitger F, von der Heydt R (1993) A computational model of neural contour processing: figure-ground segregation and illusory contours. In: Proc. 4th Int. Conf. Computer Vision pp 32-40. Washington D.C.: IEEE Computer Society Press.
• Peterhans E, von der Heydt R (1993) Functional organization of area V2 in the alert macaque. Eur J Neuroscience 5: 509-524.
• von der Heydt R, Heitger F, Peterhans E (1993) Perception of occluding contours: Neural mechanisms and a computational model. Biomedical Research 14, suppl. 4: 1-6.
• Heitger F, Rosenthaler L, von der Heydt R, Peterhans E, Kübler O (1992) Simulation of neuronal contour mechanisms: from simple to endstopped cells. Vision Res 32: 963-981.
• Rosenthaler L, Heitger F, Kübler O, von der Heydt R (1992) Detektion von ein- und zwei-dimensionalen Bildstrukturen in komplexen Grauwertbildern. In: Mustererkennung 1992 (Fuchs SHR, ed), pp 290-298. Berlin: Springer.
• Rosenthaler L, Heitger F, Kübler O, von der Heydt R (1992) Detection of general edges and keypoints. In: Proc. Computer Vision - ECCV'92 pp 78-86. Berlin: Springer.
• von der Heydt R, Peterhans E, Dürsteler MR (1992) Periodic-pattern-selective cells in monkey visual cortex. J Neuroscience 12: 1416-1434.
• Peterhans E, von der Heydt R (1991) Elements of form perception in monkey prestriate cortex. In: Representations of Vision: Trends and Tacit Assumptions in Vision Research (Gorea A, Fregnac Y, Kapoula Z, Findlay J, eds), pp 111-124. Cambridge: Cambridge University Press.
• Peterhans E, von der Heydt R (1991) Subjective contours -- Bridging the gap between psychophysics and physiology. Trends in Neurosciences 14: 112-119.
• von der Heydt R, Peterhans E, Dürsteler MR (1991) Grating cells in monkey visual cortex: coding texture? In: Channels in the Visual Nervous System: Neurophysiology, Psychophysics and Models (Blum B, ed), pp 53-73. London: Freund.
• Treisman A, Cavanagh P, Fischer B, Ramachandran VS, von der Heydt R (1990) Form perception and attention: Striate cortex and beyond. In: Visual Perception: The Neurophysiological Foundations (Spillmann L, Werner JS, eds), pp 273-316. New York: Academic.
• von der Heydt R, Peterhans E, Baumgartner G (1990) Neuronal processing of contours. News in Physiological Sciences 5: 152-155.
• Peterhans E, von der Heydt R (1989) The whole and the pieces -- cortical neuron responses to bars and rows of moving dots. In: Seeing Contour and Colour (Kulikowski JJ, Dickinson CM, Murray IJ, eds), pp 125-130. Oxford: Pergamon.
• Peterhans E, von der Heydt R (1989) Mechanisms of contour perception in monkey visual cortex. II. Contours bridging gaps. J Neuroscience 9: 1749-1763.
• von der Heydt R, Peterhans E (1989) Ehrenstein and Zöllner Illusions in a neuronal theory of contour processing. In: Seeing Contour and Colour (Kulikowski JJ, Dickinson CM, Murray IJ, eds), pp 729-734. Oxford: Pergamon.
• von der Heydt R, Peterhans E (1989) Cortical contour mechanisms and geometrical illusions. In: Neural mechanisms of visual perception (Lam DMK, Gilbert CD, eds), pp 157-170. The Woodlands: Portfolio Publishing.
• von der Heydt R, Peterhans E (1989) Mechanisms of contour perception in monkey visual cortex. I. Lines of pattern discontinuity. J Neuroscience 9: 1731-1748.
• von der Heydt R, Peterhans E, Baumgartner G (1988) Illusory contours and cortical neuron responses. In: Natural Computation (Richards W, ed), pp 70-75. Cambridge: Bradford Book MIT Press.
• von der Heydt R, Peterhans E (1988) Contour processing in primate visual cortex. In: Mustererkennung 1988 (Bunke H, Kübler O, Stucki P, eds), pp 111-127. Berlin: Springer.
• Baumgartner G, Peterhans E, von der Heydt R (1987) Neuronal mechanisms of the first, second, and third order contrast in the visual system. In: Computational Systems - Natural and Artificial (Haken H, ed), pp 35-43. Berlin: Springer.
• Peterhans E, von der Heydt R (1987) The role of end-stopped receptive fields in contour perception. In: New Frontiers in Brain Research: Proceedings of the 15th Göttingen Neurobiology Conference (Elsner N, Creutzfeldt O, eds), pp 29. Stuttgart: Thieme.
• von der Heydt R (1987) Approaches to visual cortical function. Rev Physiol Biochem Pharmacol 108: 69-150.
• Peterhans E, von der Heydt R, Baumgartner G (1986) Neuronal responses to illusory contour stimuli reveal stages of visual cortical processing. In: Visual Neuroscience (Pettigrew JD, Sanderson KJ, Levick WR, eds), pp 343-351. Cambridge: Cambridge University Press.
• Baumgartner G, von der Heydt R, Peterhans E (1984) Anomalous contours: A tool in studying the neurophysiology of vision. Exp Brain Res suppl.9: 413-419.
• von der Heydt R, Peterhans E, Baumgartner G (1984) Illusory contours and cortical neuron responses. Science 224: 1260-1262.
• Dürsteler MR, von der Heydt R (1983) Plasticity in the binocular correspondence of striate cortical receptive fields in kittens. J Physiol (Lond) 345: 87-105.
• Hänny P, von der Heydt R (1982) The effect of horizontal-plane environment on the development of binocular receptive fields of cells in cat visual cortex. J Physiol (Lond) 329: 75-92.
• von der Heydt R, Hänny P, Dürsteler MR (1981) The role of orientation disparity in stereoscopic perception and the development of binocular correspondence. In: Sensory Functions (Grastyán E, Molnár P, eds), pp 461-470. London: Pergamon.
• von der Heydt R (1979) Stereoskopische Wahrnehmung der Orientierungsdisparation. Experimentelle Untersuchung zum Nachweis eines neuen stereoskopischen Mechanismus. Thesis. Zürich: ETH, Swiss Fed. Inst. Tech.
• von der Heydt R, Hänny P, Adorjani C (1978) Movement aftereffects in the visual cortex. Arch Ital Biol 116: 248-254.
• von der Heydt R, Adorjani C, Hänny P, Baumgartner G (1978) Disparity sensitivity and receptive field incongruity of units in the cat striate cortex. Exp Brain Res 31: 523-545.