Thursday, November 13, 2008

Techical aspects about IR photography with digital cameras

I use a digital SLR camera (DSLR), Nikon D60, converted by lifepixel.com to take photos in the near infra-red (IR), i.e. about 720 nm - 1200 nm (this is NOT thermal infrared: thermal images are taken around 10000 nm). The conversion replaces the IR block filter between lens and CCD with an IR pass filter. When I look through the view finder I see "normal" colors, but what I see on the display after taking a shot is a first rendition of an infrared image.

The conversion significantly alters the response of the color receptors of the R, G, and B channels: instead of reflecting the red, green, and blue components of a scene, the combination of the IR-pass filter and the dye inside of the receptors leads to the following spectral sensitivity (my measurements were inspired by Samir Kharusi):

The response of the R, G, and B-channels are red, green and blue shaded curves, respectively. First off, we see that R, G, and B channels have different spectral response curves, i.e. we are actually able to capture with this setup something like color (the thin, light red and green lines are examples for spectral separation calculated from a linear combination of the R, G, and B channels). Most notable, the R channel has a response from 720 nm to the limit due to the bandgap of the Silicon CCD; the B channel is sensitive from 800 nm on. Note the dominance of the R-channel: any light in the near IR that hits the sensor and triggers a response of G or B channel will also trigger a response of the R channel. Hence, we will probably want to use a custom white balance: if we tell the camera to use a build-in white balance (designed for visible light, not near-IR) we'll get a photo that looks very red. We also see that the R channel is most sensitive in the shorter wavelengths while the B channels is most sensitive in the longer wavelengths. Hence, we may want to swap color channels and display the R-channel as blue and the B-channel as red.

Camera white balance

It seems that most people set a custom white balance in IR by taking a photo of green foliage (e.g. grass) instead of something white [in D60: Shooting Menu -> White balance -> Preset manual -> Measure]. This will render foliage white on a JPG straight-out the camera. I like this choice: foliage tends to have a flat reflectivity spectrum in the near IR, i.e. even if we were able to see colors in the near IR we would probably see foliage as gray or white.

Post-processing of JPGs


My program of choice for basic image manipulations is the free IrfanView. The very minimum amount of post-processing I do is swapping the R and B channels of my JPGs (Image -> Swap Colors -> RGB to BGR). Hence, we can interpret the resulting image as follows: if an object looks reddish it has an infrared color, and if an object looks blueish it has more of a visible (deep-red) than infrared tone. I also like to increase the saturation a bit to make the photo a more dramatic (Image -> Color corrections...). Here's an example:


Colors in near-IR

If we apply a white balance based on a green surface, and map the R and B channels to blue and red, respectively (as outlined above), then we tend to get the following colors:
  • white/gray: thick, translucent clouds; snow; leaves; grass
  • blue: clear ice; water; clear sky; skin
  • red: bark; dry grass; some (often black) synthetic materials; some sun glasses
Due to the contrast between snow and ice at longer wavelengths, near-IR photos (typ. > 800 nm) have long been used in science to photograph glaciers. For the same reason, near-IR photos of sea ice surfaces often reveal more structural detail. This is illustrated below; the exposed edges of ice blocks have very little red-components, i.e. they are very dark above 800 nm compared to the surrounding snow. I took these photos under overcast skies at the edge of landfast sea ice in Wales, Alaska on May 6, 2008:
Near-IR:

Visible:


JPG vs. raw image format

I usually save near-IR images in RAW format because I usually want to adjust color balance and exposure. The RAW image (Nikon .nef file) contains a thumb nail jpg which can be displayed and extracted with IrfanView. Of course it doesn't hurt to have the camera save both JPG and RAW at the same time.

Exposure setting

This is tricky with most DSLRs because light is metered independently of the CCD and filter in front of it. Hence, the converted D60 continues to meter light in the visible, and the brightness of a scene in the visible tells us little about the brightness in the near-IR. The amount of IR light increases significantly as we move from overcast skies to clear skies (it is even higher under tungsten light, and much lower under fluorescent light). In order to prevent the R-channel from saturating, I set the exposure compensation on my camera to something like 0 to -1 EV under overcast skies, 0 to -2 EV under clear skies, much lower under indoor tungsten lighting, and much higher (e.g. +3 EV or more) under fluorescent light. Actually, forget these numbers and try for yourself: it depends on the scene. Experimentation is my friend.

Optimal exposure settings depend on what's going to happen with the photo: if I am only interested in near-IR black-and-white photography then I may be most interested in wavelengths above 850 nm. In this case I will expose appropriately for the B channel and deliberately overexpose R and G channels.

Unfortunately, it is very difficult to tell from the display of the D60 whether an image is overexposed (it may be impossible). I find that I have to look at the RAW image data to tell for sure; so I always underexpose when in doubt. Underexposure does not cause artifacts although the picture is dark and there is more noise in the frame. (The RAW images of the D60 are 12 bit.)

In order to analyze the raw image file I use dcraw (free) to convert Nikon's raw NEF image to the well-documented 16-bit PGM format [dcraw -D -4 filename.nef]. I extract a crude histogram with a short C program that I'd be happy to post here if only I knew how.

Focus

The focal point of a lens depends on the wavelength. Quality lenses are designed to compensate for this from 400 nm to 700 nm; however, we use them around 900 nm. Lifepixel performs an adjustment of the focal point for a specified lens set to a specific focal length (I think they do 50 mm by default). However, we can expect to get slightly out-of-focus (soft) images when we use different equipment. Usually, I use a 18-55 mm lens and didn't notice serious issues here. However, with a 200 mm lens I have to manually adjust the focus slightly after autofocussing since the subject is obviously out of focus. (This is easy to detect in playback with the zoom function.)

Hot spot artifact

I understand that there is reflection of light between the "polished" CCD and the lens in the camera. Manufacturers make sure that this does not affect the image for visible light. However, a big ugly red spot may appear if we use an unfortunate combination of camera and lens for infrared photography. I use the following Nikon lenses without problems on my converted Nikon D60 body:

Choice of camera and lens

I was looking for a system with the following key features
  • low noise
  • not too expensive
  • free of hot spots
  • does white balance of IR images properly
The D60 bundled with the 18-55mm lens seemed to have been the best compromise back in April 2008. The CCD chip is significantly bigger than what is used in "point-and-shoot" cameras; this is where size does matter: the bigger the pixels the higher the signal-to-noise ratio. A full frame (FX) camera would have been even better but considering point 2, price, this was not an option. Seeing that the D60 came bundled with the relatively cheap and good 18-55 mm lens, and that this lens actually produces no hot spots there seemed to have been only one alternative: the D40(x). However, the D40 uses an older image processor that does not allow to define the whitebalance based on a green image...
Problem with Canon was that their lenses of suitable focal lengths happened to produce hot spots unless I was willing to pay big bucks. If I were to buy a new system today I'd take into account LiveView capabilities in today's DSLRs.