System Analysis (Top)

Digital Photography (Top)

Digital Still-/Motion-Picture Camera (Top)

The following pictures illustrate one of the challenges introduced by a digital color camera using a single image sensor. Because the scene is captured by means of photo detectors sensitive to luminance intensity, the light coming from the scene must go through color filters which separate the light into red, green and blue components an only one of which is is measured.

The limitations of the optical and sensor processing from (1) to (4) result in loss or corruption of scene information that the interpolation algorithms used in color reconstruction (5) & (12) and digital zoom (6) & (13) will not be able to recover. The example below illustrates it.

The scene is such that fine details, so fine that the sensor resolution can't catch them without zooming in, are located on the border, while coarse details are located in the center. The optical system is assumed to be an ideal one which deliver without distorsion an image whose resolution matches those of the sensor(s).

Digital Camera
with Optical Zoom and Three Sensors
Digital Camera
with Digital Zoom and Single Sensor

Let's capture the complete scene.

Let's capture the complete scene.

Because the original scene is properly filtered by the optical system to deliver an image whose resolution matches those of the sensors (i.e. that the Nyquist criterion is satisfied), and because three sensors ensure that the three primary color components are not undersampled, the sampled image does not contain any visible artefact.

When the original scene is filtered by the optical system to deliver an image whose resolution matches those of the sensor, it does not take into account the undersampling introduced by the color filters (1/4 of the sensor resolution for the red and blue components, 1/2 for the green one). Therefore the image is sampled by the sensor although it still contains spatial frequencies which do not satisfy the Nyquist criterion. The sampled image contains many visible artefacts resulting from the aliasing effect.

Some improvements are possible:

- The optical system can filter more to deliver less resolution to the sensor in order to satisfy the Nyquist criterion for each primary color component. This means less information, i.e. less fine details of the scene visible on the final image, but also less visible artefacts (if any).

- The color reconstruction algorithm can make assumptions, such as "the scene content is such that the primary color components are correlated", and benefit from the extra information in the green component in order to improve the red and blue ones and partly restore them from the aliasing effect. When the assumption is verified, the final image looks better. When it is not, it looks worse. The difficulty is to ensure that the assumption is not verified in areas of the image where the human eye is less sensitive.


Zoom 2x in the center.


Zoom 2x in the top-left corner.

Zoom 2x in the center.


Zoom 2x in the top-left corner.

When zooming in with the optical system and moving around the scene, finer details of the scene are revealed. These are details which were enlarged by the optical system prior to any filtering or sampling and whose size now matches the sensor resolution. They become visible and are properly rendered.

The digital zoom is a process which takes place after the optical filtering and the sensor sampling once the information has been lost or corrupted. It has no way to recover the information and therefore it can not improve the image quality or reveal finer details of the scene. It will only enlarge the artefacts introduced by the system. Enjoy.


Test Images for the Development of Image Processing Algorithms (Top)

As shown on the above example, simulating the image and video processing algorithms of the digital camera system is a must to evaluate how they perform and if they produce satisfying results. For such simulations, different kinds of test images are necessary: synthetic images which highlight, from the signal theory point of view, such or such behavior of processing algorithms; and natural images which show how these behaviors affect the rendering of real-life scenes.

For this purpose, the following test images may help:

Image Description

This test image visually associates spatial and frequency components in a one-to-one relationship, where each single (x, y) position on the plate corresponds to a single (fx, fy) frequency. The middle of the plate represents the 0.0 relative frequency, while its border represents the 0.5 one. The use of this image highlights the behavior of the algorithm for various spatial frequencies, i.e. for coarse as well as for fine details of a scene.

The same pattern with a focus on horizontal and vertical directions (the most common directions observed in the human environment).

This test image shows the spatial impulse and step responses of a processing algorithm, whose characteristics might results in visual artefacts. For instance, applying a 2D low-pass filter to an image does not only remove its spatial high-frequency components, smooth the luminance transitions and remove the finest details of a scene. Depending on the strength and the length of the overshoot in the filter step response, it might result in visual artefacts, such as light or dark waves or rings, near the luminance transitions of the scene.

The same pattern with a focus on horizontal and vertical directions (the most common directions observed in the human environment).

This test image shows how a processing algorithm performs in presence of noise for various levels of the luminance and for various levels of noise. The luminance increases continuously and linearly from 0% on the left to 100% on the right. The noise has 8 different levels ; the various levels corresponding to a SNR equals to +Inf, +24dB, +21dB, +18dB, + 12dB, +09dB, and +06dB. Some image processing algorithms perform poorly in presence of noise. For instance, the so-called gamma correction introduces a gain much greater than one in low-luminance areas resulting in a strong amplification of the noise in areas where it is the most visible.

The last test image gives an overview of the color behavior of a given image processing algorithm.


You can find an up-to-date list of my bookmarks related to digital photography on del.icio.us!


Stereoscopic Vision (Top)

You can find an up-to-date list of my bookmarks related to stereoscopic vision on del.icio.us!