Too Good to Be True?

If you came here by way my flickr D7000 13.5 Stops!! image you have seen that I added (12 Stops Maybe?) to the title. So what is the true scoop?  Bottom line--Old Scrib messed up a bit. About half way messed up.


The part about being able to pull out a very noisy image  even when you shoot 13 stops underexposure is true. I just didn't prove that using the screen shot I posted. Those images were, as I originally suspected and talked about in the last post, a little too good to be true. 

Here is my take number two on the Dynamic Range question.  How do I know this experiment is more correct?  If I had halved  my exposure one more time for the last post I would have discovered  that the CMOS sensor and A/D in the D7000 doesn't mess around. When it drops over the no data to digitize cliff it doesn't produce a very noisy black image.  Instead the image stays 0,0,0 no matter how much I tried to brighten it.

Picking the exposure where the brightest part of the image is just about to bloom and then counting down stops is subject to experiment error.  Counting up from total black is better. And since I did it both ways this time I'm more sure of my exposures.


The image below is 9 stops under exposed.  A corner of the white paper by the rack started to blink using ACR and a 4 second exposure. With a 1/4000 exposure  that image was totally black. The D7000 was set to take 14bit uncompressed RAW. The IS0 was set to 100, the only real ISO of the camera since any other setting only amplifies the ISO 100 image.
.
Cloudy, winter afternoon lighting.  The drapes on the windows were 80% closed.  When I spot metered the scene the strip of wall had 1/4 the lighting of the paper, the books in the closet had 1/16 the lighting.

The image is a bit noisy but remember this image was taken with 1/500 of the light needed for a "correct" exposure.  And despite the noise, it has enough color information for my WB eyedropper to do its thing and show accurate colors.

Why start off by showing a 9 stop underexposed image?  "Nine point two stops". That's the dynamic range number dpreview claims in their review of the D7000. 

Sorry dpreview folks, I beg to differ. My D7000 has much more DR than 9.2 stops. If you don't believe me take a 9 stop underexposed picture with your D7000 .

Since I am taking on the most visited photography review site on the web I'll start off by asking a question. In photography what does dynamic range  mean?

A whole bunch of different things I discovered. I spent a good part of a night in various forums, reading the true facts, half true facts, and down right silly facts that came up ever time  a newbie asked a question about dynamic range. It was sort of like asking about 'freedom and the American way." in the political forums. Folks who hang out in Internet forums are not afraid to defend their opinions--be they be right, wrong or maybe.

http://www.sprawls.org/ppmi2/FILMCON/#The%20Characteristic%20Curve


My first hit in a google search got me this curve. It is the film characteristic curve from an X-ray education site.  Since doctors study negatives not prints, the white to black scale is reversed from what we would see in a print.  With a print the light area labeled Base + Fog Density determines what is visible in the shadows just as the dark Shoulder does for the highlights

Back in the 19th century when a photographer took off his lens cap and counted his pulse beats to set his exposure. Mr. Hurter and Mr. Driffield of D&H curve fame (the old name for this curve) decided to work out a scientific explanation of exposure. At first they expected that if they doubled the light on their film they would double the darkening, but soon discovered things were more complicated. Film exposure is not linear. It is logarithmic.

The x axis is the f-stops. The y axis is the optical density.  Place a negative in an enlarger and at density 1 a tenth of the light hits the print paper. Similarly  at density 2 it's a hundredth and at density 3 a thousandth of the light.  

The straight part of the curve shows the useful exposure or dynamic range of the film. The roll off at bottom is because the film stock isn't completely transparent and because of the chemical action of the developer on unexposed film.  The roll off at top is because at density 3 the developer stops precipitating silver onto the negative. 

With film dynamic range is set by real chemical and physical limitations. Every film type has its own characteristic curve and dynamic range. Back in the film days, dynamic range disputes were a bit boring. The best anyone could fight over was how much of the curve  film manufactures should use when they calculated numbers for their spec sheets

Here is a set of curves from dpreview's D7000 'dynamic range' widget. If it looks familiar it should.  Relative exposure on the x axis, 0-255 display values--log corrected by the monitor's gamma setting-- on the y axis. The high tech version of the D&H film characteristic curve.


So what is wrong?  If  Mr. Hurter and Mr. Driffield borrowed Mr Wells's time machine and took a time trip to Best Buy in search of a digital camera for their experiments they would discover their original guess about a linear curve was right.  Double the light on a pixel, double the photo electrons and the A/D output in the RAW image. At the bottom you run out of A/D levels or in cheaper cameras signal to noise, At the other end, the pixels saturate and everything in the neighborhood goes pure white,   If your camera curve isn't linear in the middle, your camera is broken.


So why the roll off? For many reasons, starting with how the human eye/brain system works and ending with what we expect to see in a photograph (think of how odd a wildly processed HDR images looks) camera manufactures added tone curves to their jpg firmware. They do their best to mimic how a film photograph looks.

With a cheap P&S you live with the tone curve the firmware engineer gave you. Pay a few more bucks and you start having choices, vivid over normal for instance. After paying a bunch of bucks for a D7000 I ended up with more tone curve combinations than I'm every going to test.While some of the in camera jpg manipulations look neat, I am a dye in the wool RAW photographer.  I make up my tone curves as I go along. That's what RAW is all about.

The dpreview widget for the D7000, labeled "Dynamic Range", graphs how 20 different tone curves and ISO settings changes a grey scale image. No more, no less. Moreover, this only effects jpgs straight out of the camera. 

So if the idea of ever using a RAW converter makes you cringe,  the 9.2 stops or less numbers may apply. But if your thing is, like mine, low light flash-less candid photograph, there is a lot more dynamic range to be had.

The 1/500 second, ISO100, f13 exposure 11 stops underexposed image.  The WB eyedropper didn't work well so I ran the saturation slider down to zero.  I'm going to go with this image, and say the with proper post processing you can pull 11 stops underexposed detail out of the deepest shadow in an image.

But to prove a point this is the 1/2000 sec 13 stop underexposed image. Sure there isn't much detail left but what did you expect. As for the color cast, I played with adjusting the channel histograms and this is the closest to white I got before I became bored.


Coming up. 


A  week from today is all goes as planned, I should be photographing dancers during the 2011 Celebrate Youth festival. Since I'll be in the same auditoriums I worked in last year, a real world dynamic range comparison between my D7000 and D60 sounds like a fun blog post.

D7000 Dynamic Range --- 13.5 Stops!

This morning I was reading a couple postings where I encountered some of the usual Internet confusion about what should be an easy to understand concept-- Dynamic Range. I was about to toss in a few words about my take on the subject  when I decided to repeat an experiment I did 10 years ago.

Then I had just bought my first digital, a 3mp Oly 3020. P&S. A high end camera in those days. And being an optical mechanical engineer and early pixel peeper I wanted to learn how the camera's innards worked. Like how many levels there were in the camera's A/D.  I asked in the only real photography forum around, rec.photography.digital on Usenet,  Since nobody seemed to know, I worked out a simple experiment so I could  enlighten the Usenet world and start down the Path to Usenet Guru.  Ah, a simpler world then.

The experiment:  Sent up a target and a find an exposure where the brightest area was starting to clip. Then keep taking pictures as you reduced the exposure until you end up with just noise. That should be at or near the lowest levels of the A/D. And though I wasn't thinking about dynamic range at the time it gives you that number.

I got fancy back then and used a table loaded with stuff.  Today. I attached a print of an excellent test target to a bed desk  (It can be found on the Internet but I don't remember its name.)  To that I added Charlotte's collection of washable markers and a paint swatch. (Home Depot paint department,  White on White to Ink Drop)

The main image in the screen shot was taken at 1/13sec,  f5 and ISO800--3 stops away from ISO 100 which uses almost all of the levels of the D7000's 14 bit A/D.  The smaller image was shot at 1/5000sec, f10,  ISO setting of 6400, for a 7 1/2 stop under exposure to give an ISO equivalent of 1,000,000.

So, believe it or not, you are looking at a recognizable image taken 13.5 stops off the "correct' exposure.

And believe it not, I did.  That image was too good with too wide a range of tones. So I did everything all over again with the same results.

After some research  I believe several things are going on.  The R G and B channels all have different gains, something I could see using the channel mixer.  Noise dithering is smoothing out the tone steps I expected.  And both Photoshop and Irfran have some sophisticated algorithms that allowed me pull out the detail in post processing.

As for the post processing,  With the really low light images you start with a black jpg  and end up with a black conversion even with the exposure slider in ACR set at 4X.  Bring what you have into Photoshop and add a levels layer. Set the  boxes on the layer lever to 0 for black, 4 for middle grey and about 50 for the end of the mostly noise histogram.  The posted image still came out dark so I adjusted  the gamma slightly in Irfan when I resized it for posting.  This also improved the S/N since Irfan bins pixels when it resizes an image

Finally the image at the bottom was taken at minus 14.5 stops into a 14 bit A/D. As expected it shows all noise. Or almost, since there was some traces of image detail in the red channel due to the different gains.

Laura revisited

Besides the run-the-detail-sliders-down-to-zero version, RAW Therapee has three other ways to reduce noise.  The one I find most interesting is the third one down on the details tab.  It has  two sliders to control how much luminous and chromatic noise is removed .

It also has a third slider labeled gamma. That is actually a gradient filter. It cuts down the noise reduction--and loss of detail--in the bright areas of the image where it is not needed much, but it still applies strong noise reduction to the dark areas where it is really needed.

A very neat effect. Off and on screen shots below. Notice how it balances the noise from the black background with almost no loss of detail in Laura's hair

Who is Laura and why is she being revisited? A Young Shakespeare Player who I blogged about earlier. See fun-with-imagej,  November 2009

Enhanced Detail

Yesterday  the Madison Flickr Meeters went to the dome of the Olbrich Gardens in search of macro images. I started off with  my 18-105mm kit lens mounted to the D7000. It's more of a closeup lens than a macro lens so I also packed up several other lens I planned to check out.

One problem I hadn't considered propped up immediately. The humidity in dome was 100% Tropical Rain Forest; my camera and lenses which had been sitting in the car were cooled down to 100% Madison Winter Frigid. Instant condensation.

Continually wiping off the filter on top of the lens was an annoyance but not a major problem. Doing the same thing to the sensor would have been a disaster. So the lens stayed on. No real macro images but some decent flower closeup.

Contrast Disabled

 Contrast enabled

Which became even more decent when I used the RAW Therapee features I blogged about yesterday.  If you do macro photography RAW Therapee is worth looking at just for Richardson-Lucy deconvolution. (There has to be a better name than "Contrast by Detail Layers.")

Am I about to toss out Photoshop? No. I shot with an off-camera flash but in this image there were a few small areas that reflected the flash straight into the camera and created annoying hot spots. So I imported the jpeg into PS and used the healing brush to clean up the image. Can't have everything in a RAW converter.

Edit-When I posted the image on flickr and viewed the black box original I saw that the image looked noisy. Setting the fine slider to 2.2 did a SUPER GREAT job of bringing out and amplifying  the noise in the image! Setting it to zero cleaned the image up. Something to remember if you use the feature.

Smiling Girls-With a Dash of RAW Therapee

I could have converted this underexposed color image into a correctly exposed B&W image using Adobe Camera RAW and Photoshop but it would have been a multi step process.  After I converted the RAW data into a RGB image I would have had to imported it into Photoshop, converted it into a LAB image and then used a curve or level layer on the L channel to adjust the exposure. With RAW Therapee all than can be done in one simple step.

When I first started out I thought, like I suspect most user do, that Photoshop did its mathematical calculations in the  RGB mode. But it starts out in the LAB mode. Instead of the familiar Red, Green and Blue channels LAB has, not surprising,  L, A and B channels. The L or Luminosity channel has all the black and white picture information.  The A channel has all the red and green color information. And the B channel has all the blue and yellow color information,

A major and practical advantage of this mode is you can apply a curve layer to the L channel and do what you want without touching the color information and introducing a color cast.  You can also do the same thing to the A and B channels to correct a color cast.  Or you can go hog wild and create colors that nature never intended.

t 

Besides LAB curves there are several other things I like abut the exposure tab in RT. If you click on the + sign on the bottom you can pull up multiple viewing boxes. When I processed the image I monitored  the eyes of the two girl, the shirt and the writing on the folder as I adjust the L channel. I've also ran the saturation down to zero for B&W. If I had wanted to change the intensities of the the various colors I could have adjusted the A&B channel. Or I could have gone to the color tab and used RT's channel mixer. There are often several ways to adjust a RAW image in RT.

(With more to come. Yesterday I saw postings about a new tone matching feature for HDR work on RAW data)

In Lab curves you can pull the the usual spot and parametric adjustments but I prefer using the control cage adjustment. This uses tangential  Belzer controls.  You don't have annoying dips at one end of a curve adjustment when you raise the other end. You also have a well designed histogram window up at the top where you need it instead having to call up a separate window.

A second feature you won't find anywhere except pricey astrophotography programs is Richardson-Lucy deconvolution. This algorithm was developed by NASA to deconvolute or enhance the images from the Hubble telescope. 

Enhances  Contrast Off