INTRODUCTION

The subject of color constancy has become more important recently due to the interests in robotic vision and electronic cameras. The need has been to understand color constancy in human vision so similar techniques can be introduced into these devices.

A variety of discussions of this subject have appeared on the WEB in recent times. Most of these have lacked any theoretical foundation related to the actual mechanisms of human vision resulting in color constancy. Most of these have also failed to define the illumination range over which the proposed mechanism applies.

Based on the recently completed book, PROCESSES IN BIOLOGICAL VISION, the phenomena of color constancy and the underlying mechanisms causing it can be defined in considerable detail. The overall phenomena is discussed in Section 17.4.5 of that work. The underlying mechanisms are discussed in Chapter 12 and Section 17.1.1.

Contrary to the intimation or claim in some of the literature and WEB papers, color constancy does not involve cognitive processing or any de-convolution in the spatial or spectral domains to determine the reflectance coefficient of a surface. It does involve temporal frequency filtering within the photoreceptor cells of the eye.

The same techniques used in the visual system to achieve color constancy have been in use in industrial and military television equipment since the 1950's.

BACKGROUND

Color constancy is closely related to the phenomena known as reciprocity failure in photographic film technology. In a sense, it is the complement to reciprocity. In human vision color constancy is directly related to the ability of the system to work over an extended radiant intensity range. Both phenomena and the transient known as dark adaptation are all controlled by the adaptation amplifiers found within each photoreceptor cell of the retina. This amplifier has been described in detail in the above text.

Some of the terminology used in the following paragraphs may be unfamiliar to some readers. A short glossary is provided at the end of this paper.

The Individual Adaptation Amplifier

The morphologically defined photoreceptor cell contains three major components; the Outer Segment, the Inner Segment, and the Neural Segment. Each segment performs a separate and distinct role in vision. The Outer Segment is not an internal part of the cell. It is an external structure associated with the cell and formed by the Inner Segment of the cell by secretion and extrusion. When coated with the appropriate chromophore of vision, the Outer Segment becomes a passive transducer of radiant energy into energy of excitation within the structure. This transduction process is inherently linear.

The energy of excitation is transferred to the neural portion of the cell by a mechanism very similar to that found in a photo-transistor. This process will be called translation to differentiate it from the above transduction process. The translation process is linear in the first order but does involve some second order non-linearities that do not impact this discussion. The dendritic structure associated with the Neural segment of each photoreceptor cell is a distributed, open base, electrolytic semiconductor device known as an Activa. The Activa is a biological transistor. This device is arranged in a differential pair type of circuit with a second Activa that acts as a distribution amplifier and a current to voltage convertor. The open base input Activa is known as the adaptation amplifier because of its unique performance capabilities.

The adaptation circuit [Sections 8.6, 12.5 & 17.1.1] is unique in three respects:

1. Adaptation is controlled by the percentage of the chromophore material associated with all of the disks of a specific photoreceptor that is electronically excited at a given time. This excitation is observable by measuring the degree of bleaching of the retina from its fully dark adapted condition. The excitation level is reduced by the average current flow from the base of the adaptation amplifier into the liquid crystalline chromophore surfaces. The average current flow from the base is a precise fraction of the current through the collector of the adatation amplifier.



2. The load impedance in the collector lead has a time constant that is finite, between 0.6 and 3.0 seconds depending on the species. The power supply supporting that load impedance is very poorly regulated. It exhibits time constants of 2 minutes and 10 minutes.



3. The output signal from the adaptation amplifier is taken from the emitter terminal.

Because of characteristics 1 and 2, the chromophoric mass associated with the input of the adaptation amplifier exhibits a very large variation in absorption cross-section. This variation is the primary cause of the loss in sensitivity at high stimulus levels.

Because of characteristic 3, the overall circuit exhibits a large amount of negative internal feedback.

The Individual Distribution Amplifier

The distribution amplifier, also located within the Neural Segment of the photoreceptor cell, is configured as a "grounded base" amplifier. In this configuration, it offers a unity current gain between the current at its emitter and the current at its collector.

The combined amplifiers of the Photoreceptor Cell

The complete circuit diagram of the photoreceptor cell is shown below.

The detailed discussion accompanying this figure is available in Section 10.10.7. By placing the adaptation amplifier in a differential pair configuration with the distribution amplifier, an additional feature is introduced. This configuration is designed to maintain a constant current through the common emitter(s) to ground load (2). As a result of this feature and the time constants discussed earlier, a very high degree of negative internal feedback is introduced into the overall photoreceptor cell circuit at low frequencies. The average current in the load attached to the distribution amplifier collector (4) remains essentially constant. At higher frequencies, the circuit exhibits considerable gain. The effect is to normalize the output signal DC gain while providing a variable AC gain that can be very large at low input signal levels. The variation in AC gain is from 1:1 to about 3,500:1 in a typical photoreceptor.

The above variation in gain as a function of input illumination forms the basic mechanism of both adaptation to changes in input intensity level and to the phenomenon of color constancy.

The spatial and spectral configuration of the visual system

The human retina consists of a mosaic of three interdigitated arrays of spectraly selective photoreceptors. The detailed parameters of each of these arrays are not well documented. However, the effect is to provide a mosaic that samples each of the the important features of the scene in each of three separate spectral regions. Each of these samples is converted into an electrical signal by a separate set of spectrally selective photoreceptor cells.

All spatial signals are converted to temporal signals for transmission from the retina to the brain. In this process, all fine image detail is converted to a temporal frequency range that insures that the resulting signal will pass through the adaptation process successfully without degradation.

The Dynamics of the visual system

To understand the dynamics of the visual system, it is necessary to review the way in which the photoreceptors in a given location of the retina are interconnected. A variety of simple symbolic representations of each adaptation amplifier are possible. The symbolic representation shown for the three adaptation amplifiers is for illustrative purposes only.

For purposes of this discussion, the signal from the transducers will be considered a voltage instead of a current because most people find it easier to think in terms of voltages. It shows an external feedback loop that does not exist in the actual amplifier. In this figure, signals are acquired from the same region of the scene imaged on the retina by the three transducers of the Outer Segments. The signals from the individual transducers are applied to the individual adaptation amplifiers as shown.

The capacitor C_A and the impedance R_eq form a low pass filter in the feedback path from the output of each amplifier to the negative, or inverting input on the left of each amplifier. This circuit has the effect of holding the DC voltage at the output of each amplifier at a constant voltage regardless of the voltage applied to the input from the transducer. However, the AC gain between the transducer signal and the signal going to the distribution amplifier may be quite high under this condition. In the case of vision, the instantaneous gain of the amplifier is a function of the voltage applied to the collector (c) of the amplifier. For large increases in the input signal, the gain of the circuit will decrease. Because of these two distinct processes, the output signal to the distribution amplifier has a constant average value and a AC signal that deviates from the average by a constant amount for a given input contrast. Each spectral channel of the visual system, and all subsequent perceptual channels operate in this fixed amplitude condition as long as each of the adaptation amplifiers is within its operating range. This range extends from a nominal minimum gain of 1:1 up to a maximum of 3500:1 and defines the photopic region of vision.

Figure 17.1.1-2 of the text is shown here. It discusses the apparent brightness as a function of illumination level in human vision. The dashed line indicates the average brightness perceived by the individual from complete darkness to the point of pain. The remarkably wide flat expanse of this figure is primarily the result of adaptation within the photoreceptor cells.

This figure is representative of the situation in the absence of color changes in the illumination. The color changes shown in the figure are due to secondary processes within the system. As discussed below in detail, if the color temperature of the illumination changes, or the reflectance of the scene shows a predominance in one region of spectral absorption by the photoreceptors, the situation changes considerably to maintain "Color Constancy."

Initial condition of the adaptation amplifiers

This paragraph is only conceptual. There is not enough data at the current time to confirm it in detail. However, it is approximately correct. The initial operating gain of each adaptation amplifier appears to be set as if it expects the input illumination to have an equal flux per unit wavelength distribution across the visible spectrum. This condition is produced by a nominal blue sky absent direct light from the sun, i. e, highly scattered sunlight. This illumination can be represented by a color temperature of 7053 Kelvin although it has frequently been described as equal to a color temperature of 6500 Kelvin. These values are not significantly different absent a specific criteria. It appears that when the eye is completely dark adapted, it is optimized for the 7053 Kelvin scene spectrum.

Operation of the adaptation amplifiers in unison

When the eye is exposed to a 7053 Kelvin color temperature scene and the intensity of the scene is varied, the [luminous intensity function] of the eye is obtained. This function exhibits a broad area of constant perceived brightness for a variation in input intensity of about five orders of magnitude. This wide range is accounted for through the operation of the iris and the adaptation amplifiers. The Iris provides a factor of 16:1 out of this total range and the adaptation amplifiers operating in unison account for the other factor of 3500:1. The hyperopic region is defined as that region of illumination higher than these two mechanisms can accommodate. Two regions are defined below the photopic region. The mesopic region is an area where all of the adaptation amplifiers are operating at maximum gain and the iris is fully open. This is an area of reciprocity failure due to secondary processes in the L-channel, and a decreasing ability to perceive both color saturation and hue due to signal to threshold level considerations. It is also an area where the conventional Principle of Univariance fails in the L-channel. Below this level is the scotopic level. It is defined by the complete absence of visual perception in the L-channel of vision relative to the S- and M-channels.

Differential operation of the adaptation amplifiers

Color constancy is present throughout the photopic region but it generally goes unnoticed until the average luminosity of the scene, over a specific spatial angle, can no longer be represented by a color temperature of 7053 Kelvin. When this occurs, each adaptation amplifier attempts to automatically compensate for the difference in average luminosity level sensed by its transducer. The result is different gains in the amplifiers of each spectral region of vision. Under these conditions, the perceived signal levels within the visual system from the pedicels of the photoreceptors to the higher centers of the cortex remain constant and the chrominance channels of the visual system continue to report the same colors relative to the fine detail in the scene (with minor changes to be discussed below). This effect can be visualized if the conditions in the scene, the gain parameters of the adaptation amplifier and the perceived signal levels in the visual system are examined. The following figure illustrates the perceived variance in brightness versus color temperature.

If the average small area radiance of the scene is represented by a 7053 Kelvin source, the nominal gain of each of the adaptation amplifiers is the same as indicated by the solid lines in the upper two frames. Under this condition, the perceived response as a function of wavelength remains a relative constant as shown in the lower frame. If the color temperature of the radiance is lowered as suggested by the doted line, the level of irradiance detected by each transducer is the integral of the absorbed flux. The value of this flux causes each adaptation amplifier to change its gain to compensate for this change in irradiance as shown in the middle frame by the dots at the centroidal wavelength of the chromophores. The dotted line is drawn through these three values merely for illustration. Following this compensation, the relative perceived signal levels in the visual system remains at a constant level at the centroidal wavelengths of the chromophores as shown in the lower frame. The dotted line is only conceptual, the absolute perceptual level as a function of wavelength is actually given by the luminous efficiency function. The dash-dot lines of the three frames show the example of a higher color temperature irradiance.

Combined group and differential adaptation

Group adaptation (related to a change in overall intensity) and differential adaptation (related to color temperature of the irradiation) can occur simultaneously as long as all adaptation amplifiers remain within their individual operating dynamic ranges. The impact of an adaptation amplifier reaching its maximum gain, and entering its scotopic region, is easily seen with respect to the S-channel. As the color temperature of the light source is reduced and the intensity level is also reduced, the S-channel adaptation amplifier reaches maximum gain first. This condition, typified by artificial incandescent illumination compared to mid day solar illumination, results in a lack of perception of blues. (Always illuminate your paintings brightly with a high color temperature source.)

Machine Vision and Color Constancy

The methods of adaptation used in vision are available for use in robotic and other man-made imaging systems. In fact, they have been widely used in such systems for many years. The author implemented such designs during the 1950's in vidicon based television equipment for the military and industry. He was awarded a patent on a charge coupled device (CCD) based imaging system in the 1970's that provided automatic exposure control over a wide range. When used in a multi-CCD array for color imaging, it was quite capable of providing color constancy over a range of at least 1000:1 without support from an iris. In the case of the vidicon cameras, both simple resistors and active impedances were used to achieve the same type of poor or spongy power supplies as used in the biological systems. In the CCD's, varying the integrated signal in devices containing a gate designed for exposure control provides the same capability at low cost.

Glossary

AC--A term derived from alternating current and used to describe the portion of an electrical waveform that changes rapidly with time.

AC coupled--Two circuits that are connected so they only pass AC signals.

AC gain--The gain of an amplifier with respect to rapidly changing signals.

DC--A term derived from direct current and used to describe the portion of an electrical waveform that does not change rapidly with time.

DC coupled--Two circuits that are connected so they pass both AC and DC signals.

DC gain--The gain of a circuit with respect to slowly changing signals.

Feedback--Refers to a process where at least a portion of a signal is returned from the output to the input of a circuit.

External feedback--A process where an external physical signal path extends from the output to the input of an amplifier.

Internal feedback--A less obvious form of feedback, typified by a common impedance element appearing in both the input and output circuits of an amplifier. There is no explicit feedback path around the amplifier.

Gain--The amplification factor, or ratio of the output signal amplitude to the input signal amplitude of an amplifier.