The Retina - Edge Detection

The anatomy of the eye consists of multiple components. Using the diagram on the right (Figure 1.1) as reference, we have:

The most important ones for the retina simulation are bolded, while the rest are for anatomy documentation purposes.

When light travels into our eyes, it passes through the cornea (8) - a clear, dome-shaped covering at the outermost of our eyes. After that, light travels through our pupils (6) - a hole that allows light to travel through. The amount of light allowed through the pupils are controlled by the iris (9). The light then goes through the lens (10, 11) - a convex structure that focuses light. After that, light hits the retina (30), triggering a phototransduction cascade, converting light signals into electrical impulses. Those electrical impulses then converge at the optic nerve (22) and travel to the brain.

We will be focusing on the retina (30) for this simulation.

Let's zoom in on the retina (Figure 1.2). The retina is composed of layers of cells. Those cells carry out different functions that essentially does two things - to convert light signals into electrical impulses and improve the quality and definition of those impulses. Here's the list of the layers and structures inside the retina. As usual, the important ones are bolded.

Rods (R) and cones (C) are responsible for converting light signals into electrical impulses (phototransduction). The rest of the cells - horizontal cells (H, HC), bipolar cells (Bi, BPC), and amacrine cells (A, AC) - are responsible for improving quality and definition (sparsification and directional selectivity). Retinal ganglion cells (G, RGC) summarizes the inputs from all those cells at a local level and send the electrical signals into our brain for conscious processing.

Here's an interesting side note: When light hits the retina, it actually hits the RGCs first, then the BPCs, all the way to the rods and cones. The rods and cones then undergo phototransduction to transmit electrical signals back to the RGCs. Because of this very weird arrangement of the retina, there is a small part of the retina that is replaced by the optic nerve so that the optic nerve can exit the eye and continue the visual pathway. As a result, we have a blind spot, but our brain compensates that blind spot fairly well so we won't notice.

In the visual pathway, after the signals leave the RGCs in the optic nerve, it goes through (Figure 1.3) the optic chiasm, the optic tract, the lateral geniculate body (LGN) in the thalamus, and the optic radiation all the way to the primary visual cortex (V1), which further connects to higher visual areas (e.g., in no specific order, V2, V3, MT, IT, MF, AL, etc.).

At and beyond the visual cortex, this is where orientation selectivity, direction selectivity, color processing (e.g., color constancy), face recognition, etc. are further processed.

There are a lot of biochemistry behind each cell in the retina. One of the most well-known biochemical mechanisms in the retina is phototransduction in the rods and cones. Of course, other cells, like HCs, BPCs, ACs, and RGCs have their own biochemical mechanisms working behind the scenes. So, the purpose of this section is to elaborate on those mechanisms.

2.1 Biochemical Basis of Signal Transduction in Retinal Cells

Cell activity can more or less be generalized into a gradient ranging from -1 to +1, with -1 being maximally inhibited activity, +1 being maximally excited activity, and 0 being spontaneous activity.

Imagine the cells are machines that take in one or more inputs (cell activity from previous connections) and shoot out one singular output (cell activity - a value between -1 and +1). We can essentially wire them in such a way that the RGCs can only respond to a specific pattern of light hitting the rods or cones.

Each cell in the retina has 3 components that make up the machine (Figure 2.1).

As we go through this section, we will be introducing different types of NT receptors, what those receptors do, and different NTs that can be released by the cell. And by learning these different types of receptors, mechanisms, and NTs, we can understand how different cells respond to inputs, process those inputs, and convert them into outputs.

2.2 Phototransduction in the Rods and Cones

Rods and cones are special because they are the only cells in the retina that take photons presence as inputs. For others, they only take neurotransmitters as inputs. Rods and cones then convert photon activity (-1 and +1) into electrical signals (-1 and +1, not a gradient).

The 3 components of rods and cones are the following:

For the receptors, rhodopsins (rod opsins for rods) and cone opsins (for cones) are responsible for photon detection, while transducin initiates the mechanism based on their activities. Not surprisingly, rhodopsins and cone opsins respond to different scenarios involving photons.

• Rhodopsins are sensitive to the number of photons, but are insensitive to the energy contained in each photon, that is, the wavelength of the light. As a result, rods are sensitive to brightness but are insensitive to color.

• Cone opsins are sensitive to the energy contained in each photon (the wavenlength of light), but are insensitive to the number of photons. As a result, cones are sensitive to color but are insensitive to brightness. This color sensitivity of cones can be further divide them into 3 types.
  • L-cones are sensitive to light with long wavelengths, like red light.
  • M-cones are sensitive to light with medium wavelengths, like green light.
  • S-cones are sensitive to light with short wavelengths, like blue light.

In a way, receptors also function like filters. For example, in the presence of blue light, L-cones will not do anything, no matter how blue the light is. However, if it detects red light, it will process that input. Likewise, if there is yellow light (a combination of red and green light), both L- and M-cones will react.

In the presence of the "correct" light, after transducin - a GPCR - receives the signals from rhodopsins or cone opsins, a subunit dissociates from transducin. That subunit inhibits an enzyme called phosphodiesterase (PDE). This, in turn, breaks down cGMP, decreasing cGMP amount in the cell. Without cGMP, special ion channels called cyclic nucleotide-gated channels (CNGC) are closed, barring Na+ from entering the cell. Since Na+ is blocked from entering the cell, the cell hyperpolarizes (Figure 2.2).

Rods and cones release glutamate when depolarized. However, since they hyperpolarize in the presence of the "correct" light, there is no neurotransmitter release.

So, in the end, an input of +1 (presence of the "correct" light) leads to an output of -1 (hyperpolarization). On the other hand, an input of -1 (lack of the "correct" light) leads to an output of +1 (depolarization).

In the context of our machine model, rods and cones essentially flip the sign of the inputs to yield an output (Figure 2.3). However, when it comes to color perception, it gets more complicated - we'll talk about it when we get there.

2.3 GABAergic HCs and ACs

HCs and ACs have the following 3 components:

Ionotropic glutamate receptors (iGluR) are ion channels that open in the presence of Glu. When the channel is open, Na+ and Ca++ enters the cell, depolarizing the cell. This process of converting chemical signals into electrical signals is called transduction (different from phototransduction, where light signals are converted into electrical signals).

The cell, when depolarized, releases gamma-aminobutyric acid (GABA).

So, in the context of our machine model, HCs and ACs essentially maintain the sign of the input. If the previous cell they connect to depolarizes, HCs and ACs will depolarize as well (Figure 2.4).

2.4 ON and OFF BPCs

Here are the 3 components for BPCs.

We discussed iGluRs before - they are ion channels that open in the presence of Glu, allowing Na+ and Ca++ entry into the cell, depolarizing it in the process. Metabotropic glutamate receptors (mGluR), on the other hand, have the exact opposite effect. mGluRs are a type of GPCRs, so in the presence of Glu, mGluRs activates a G-protein, which then will eventually lead to the opening of specific ion channels that do one thing - allow K+ to leave the cell. Since K+ are leaving the cell, the cell hyperpolarizes.

BPCs release Glu when depolarized.

Here's the interesting part. There are 2 types of BPCs - ON BPCs and OFF BPCs. ON BPCs have mGluRs as their receptors while OFF BPCs have iGluRs as their receptors. As a result...

So, in the context of our machine model, ON BPCs will flip the sign of the inputs, while OFF BPCs will maintain the sign of the inputs (Figure 2.5).

2.5 Algorithmic Determination of RGC Activity

Before we talk about the algorithm, let's quickly look at the components of the RGC.

So, how can we algorithmically determine the activity of RGCs using our machine model? The answer lies in mixing and matching all those different cells in certain ways. It's like forming a circuit of cells - the circuit will have different functions depending on how we wire the circuit.

Here are some ground rules for how we should wire our circuit:

This somehow flexible ruleset gives us a lot of possibilities of circuits (e.g., there is no rule against feedback inhibition). Feel free to experiment!

Using this, let's look at the simplest circuit:

circuits 1a and 1b basically boils down to the following:

Circuit	Light	Rod	BPC	RGC
1a (Rod -> ON BPC -> RGC)	+1	-1	+1 (ON)	+1
	-1	+1	-1 (ON)	-1
1b (Rod -> OFF BPC -> RGC)	+1	-1	-1 (OFF)	-1
	-1	+1	+1 (OFF)	+1

The circuits will get more and more complicated as we go on. For example, the circuit we will discuss in edge detection have 5 major elements.

So, we have the ability to form circuits. Depending on how we construct the circuit, RGCs will behave differently. Evolution has helped us develop a good few circuits in the retina, with each circuit refining our vision definition. For example, this section is about edge detection (sparsification) - a type of retinal circuit that specializes in detecting edges or boundaries of an object.

Here, we introduce the concept of receptive fields (RF).

Now, let's introduce 2 new circuits.

Similarly, the last 3 elements in this circuit are basically from circuits 1a and 1b. There are 2 differences though - our new circuits have 2 extra elements (surround rods and HCs) and new adjectives (center and surround). See Figure 3.2 for a diagram of circuit 2a.

Knowing that HCs are GABAergic, the connected center rods can be hyperpolarized by HCs depending on surround rods activity. Let's use circuit 2a as an example.

Circuit	Scenario	Light	Rod	HC	BPC	RGC
2a	Center Response	+1	-1		+1 (ON)	+1
		-1	+1	+1
	Surround Response	-1	+1		-1 (ON)	-1
		+1	-1	-1
	Both Response	+1	-1		0 (ON)	0
		+1	-1	-1
	Neither Response	-1	+1		0 (ON)	0
		-1	+1	+1

For circuit 2b, simply flip the signs of values in the BPC and RGC columns, since the only difference between 2a and 2b is the ON and OFF BPCs.

Circuit	Scenario	Light	Rod	HC	BPC	RGC
2b	Center Response	+1	-1		-1 (OFF)	-1
		-1	+1	+1
	Surround Response	-1	+1		+1 (OFF)	+1
		+1	-1	-1
	Both Response	+1	-1		0 (OFF)	0
		+1	-1	-1
	Neither Response	-1	+1		0 (OFF)	0
		-1	+1	+1

For a better understanding of this concept, the simulation at the top of this webpage provides us with a visualization of circuit 2a under all 4 scenarios: center, surround, both, and neither responses. There is also a custom response option available, which allows us to drag the circle around to see how it affects retinal cell activities.

Once we got ourselves acquainted with this edge detection algorithm of sorts, let's take a step back and generalize this concept. This algorithm essentially divides the receptive field into 2 regions - a center RF and a surround RF. The center rods still do their jobs as usual, as detailed in circuits 1a and 1b. The surround rods, on the other hand, inhibit center rods via HCs when they detect light. This mechanism involving surround rods inhibiting center rods via HCs when light is present is called lateral inhibition. The RGCs also get names. RGCs in circuit 2a are called ON-center RGCs, while RGCs in circuit 2b are called OFF-center RGCs.

Using the ON- and OFF-center RGC circuits (Circuits 2a and 2b, respectively), we can explain certain visual illusions.


4.1 The Hermann Grid Illusion

The Hermann grid illusion (Figure 4.1) consists of a black grid on a white background (or vice versa), forming a series of intersections. When looking at the grid, faint grayish spots appear at the intersections, but they vanish when directly fixated on. The illusion creates the false perception of fluctuating dark spots that aren't actually present, making it seem like the intersections are darker than the rest of the white spaces. This effect is strongest in peripheral vision and disappears when focusing on a specific intersection.

• Question 1: Why are there gray dots at the intersections in our periphery?

The answer lies in our edge detection (sparsification) circuit. Let's look at some receptive fields at our periphery when we look at the grid.

Figure 4.2 describes receptive fields that are responsible for the gray dots in our periphery. Observe receptive fields 1 and 2 and assume ON-center RGC circuits.

• RF1: There is an excited center RF and a relatively more excited surround RF. As a result, there will be more lateral inhibition, leading to a less depolarized RGC.
• RF2: There is an excited center RF and a relatively less excited surround RF. As a result, there will be less lateral inhibition, leading to a more depolarized RGC.

RF	Center RF Rods	Surround RF Rods	Lateral Inhibition	RGC
RF 1	Excited	More Excited	More	Less Depolarized
RF 2	Excited	Less Excited	Less	More Depolarized

Since RF2 has less lateral inhibition than RF1, RF1 will present a slightly darker shade compared to RF2, giving the intersection a faint grayish spot compared to the white lines.

Similarly, for the white grids on black background version, we have Figure 4.3 with receptive fields 3 and 4 (Still assume ON-center RGC circuits).

• RF3: There is a non-excited center RF and a relatively less excited surround RF. As a result, there will be less lateral inhibition, leading to a more depolarized RGC.
• RF4: There is a non-excited center RF and a relatively more excited surround RF. As a result, there will be more lateral inhibition, leading to a less depolarized RGC.

RF	Center RF Rods	Surround RF Rods	Lateral Inhibition	RGC
RF 3	Not Excited	Less Excited	Less	More Depolarized
RF 4	Not Excited	More Excited	More	Less Depolarized

Since RF3 has less lateral inhibition than RF4, RF3 will present a slightly lighter shade compared to RF4, giving the intersection a faint grayish spot compared to the black lines.

• Question 2: Why do the gray dots disappear when we look at the intersections directly?

This is easier to answer. Since our receptive fields become smaller as we go closer to the fovea in our retina, when we center our vision, the receptive fields actually don't cover a lot of area (Figure 4.4). As a result, the RGCs are all under spontaneous activity.


4.2 The Mach Bands

Mach bands is an optical illusion named after the physicist Ernst Mach. It exaggerates the contrast between edges of the slightly differing shades of gray, as soon as they contact one another, by triggering edge detection in the our visual system (Figure 4.5).

Figure 4.6 describes a common Mach band illusion, where an illusory "X" appears in the large image to the right.

• Question 1: Why is there an illusory "X" in the large image?

Let's zoom up to the corner of the gradients and compare receptive fields as usual (Figure 4.7). Observe receptive fields 5 and 6 and assume ON-center RGC circuits.

• RF5: There is an excited center RF and a relatively less excited surround RF. As a result, there will be less lateral inhibition, leading to a more depolarized RGC.
• RF6: There is an excited center RF and a relatively more excited surround RF. As a result, there will be more lateral inhibition, leading to a less depolarized RGC.

RF	Center RF Rods	Surround RF Rods	Lateral Inhibition	RGC
RF 5	Excited	Less Excited	Less	More Depolarized
RF 6	Excited	More Excited	More	Less Depolarized

Since RF5 has less lateral inhibition than RF6, RF5 will present a slightly lighter shade compared to RF6, giving the intersection a faint white gradient compared to the gray background.