Shoot at ISO21000--Print at 30 by 45 inches

Sounds insane.  ISO21000 noise. Huge print size.  Welcome to the world of RAWTherapee noise reduction.

Rhianna is letting the world know she just bowled a strike.

The gory photo details. The camera was set to ISO6400-- which is just a conventional way of saying the D7000's sensor's true and only ISO100 signal is preamp boosted  by a factor of 64 before it enters the A/D.    Since the photo had to be boosted again by an exposure compensation of 1.8EV to shift the red histogram I multiplied the camera's ISO by 3.24 (1.8 squared)  to get a correct noise IS0 of 21000.

Ain't a cheating fudge factor. That's how sensor physics works.

Now the triumph of  RT noise reduction. But first here is what not to do.

I set the exposure compensation correctly before I went to the CIECAM02 tools to improve the color, contrast and lightness. But I forgot to return to exposure to readjust the histograms  When I caught my mistake, I checked the red color channel ( box at top) for blooming (red areas).  Because I liked my CIECAM02 fine tunings,  only one channel was effected  and the blooming area areas weren't in any critical part of the image (see http://scribble-jpc.blogspot.com/2013/02/rescuing-really-important-snapshots.html ) I decided to go with my mistake. After all I was getting a WOW Signal to Noise of 20!

Wrong move.  After wasting a bunch of time and discovering a few nagging inconsistencies  I corrected the exposure comp.  One measurement later and I discoverer the real double WOW WOW S/N was to be 28!!

A few points on experimental techniques. I took the noise line profile off Jilly's arm because that area has constant color--the curves are flat-- and the arm is close to the center of the tonal range--95 in a range of 0-255. These curves have to come out flat or don't bother doing the extra work to come up with a numerical signal to noise.  You will be measuring bumps and slopes.  Not the noise.

Looking more closely I should have shortened the length of the line profile of the last measurement  to get rid of the bump at the beginning. That error added about 0.5-1 to the last StdDev (standard deviation) number.  That's why I averaged 7 measurements to clean up these errors. Even without these moments of carelessness, remember we are dealing with noise. Three measurement are a minimum, 7 to 9 are better and a 100 measurements  would be an insane overkill.  We are doing a square law experiment and the Law of Diminishing Returns kicks in with the second measurment

Plus what's so wow about a S/N of 28. Why not 280 or even 2800. If you got  the bucks to buy the camera, like the Hubble Space Telescope or the Kepler Cosmic Background Radiation Telescope in the news recently, you can go for those high S/N ratios.  But for the rest of us, with this image taken under these exposure conditions, a S/N of 28 does equal a 30X45 poster pined to a bedroom wall.

With modern tech we can pixel-peep (and complain about what we see) far too easily. Rhianna's face is 8 inches wide on my super calibrated monitor (something that the world can't see --- for the complaint  http://scribble-jpc.blogspot.com/2013/02/in-search-of-perfect-eyeball-monitor.html ) . It is 3 inches on a 11 by 14 print I had made using the older 4.0.9.185 noise reduction.  Do the math and this is what you will see on a 30 X 45 inch wall poster.

That  print had a S/N of ~15.  If you were looking for noise there was a hint of it on Rhianna's face and in the brown area under the big number one in the background. But it's nothing that jumps out at you. I sure Rianna's mom won't complain when I give her the print next time I'm over in her part of the city.

EDIT--If you looked at this post before you might notice that Jilly has suddenly morphed into Rhianna.
Rhianna is Jilly's stepsister. They both live across the street from Charlotte. I got their names mixed up. Sorry girls. :-(

After I posted this I remember I do have a big display my flat screen LCD TV with its port in the side for usb sticks and memory cards.  When I displayed this image big it looked just like I said it would from my calculations.

The details from the pp3. I started with the HighISO profile. The exposure  comp of 0.88 I set. I may have fiddled with that other setting. Or they may have come with the profile.


[Exposure]
Auto=false
Clip=0.02
Compensation=0.88
Brightness=2
Contrast=22
Saturation=5
Black=332
HighlightCompr=0
HighlightComprThreshold=33
ShadowCompr=50
CurveMode=Standard
CurveMode2=Standard
Curve=0;
Curve2=0;

The image needed some sharpening. Except for 'only edges' the setting are the defaults. If it wasn't for the 'only edges' algorithm I never would have dared using sharpening on a high noise image.

[Sharpening]
Enabled=true
Method=usm
Radius=0.5
Amount=125
Threshold=20;80;2000;1200;
OnlyEdges=true
EdgedetectionRadius=1.8999999999999999
EdgeTolerance=1800
HalocontrolEnabled=false
HalocontrolAmount=85
DeconvRadius=0.75
DeconvAmount=75
DeconvDamping=20
DeconvIterations=30

White balance off Jilly's shirt

[White Balance]
Setting=Custom
Temperature=3058
Green=0.80500000000000005

The color and appearance fine tuning

[Color appearance]
Enabled=true
Degree=100
AutoDegree=true
Surround=Average
AdaptLum=16
Model=RawT
Algorithm=JC
J-Light=27.100000000000001
Q-Bright=-1
C-Chroma=-5
S-Chroma=0
M-Chroma=0
J-Contrast=44.5
Q-Contrast=64.5
H-Hue=0

The HighIso defaults. I could have fiddled with them for a better S/N but why bother. My 30 inch printer-imaginary- is out of ink-imaginary- and will stay that way given the cost of feeding a real printer. For five bucks and change I can get a 12 by 18 inch print from my local Woodmans-one that matches the image on my screen. As long as I switch the room lights back ovet to 5000K spots. Perfect color management ain't easy.

[Directional Pyramid Denoising]
Enabled=true
Luma=50
Ldetail=50
Chroma=50.689999999999998
Method=Lab
Redchro=0
Bluechro=0
Gamma=1.7


Find RAWTherapee  4.0.11.1 here
http://www.visualbakery.com/RawTherapee/Downloads.aspx

For an excellent tutorial on noise by Emil, the guru behind RT's new noise reduction systen
 http://theory.uchicago.edu/~ejm/pix/20d/tests/noise/noise-p2.html

Rescuing REALLY IMPORTANT snapshots from overexposure

Last summer Charlotte took her mother, her grandmother and me on a time travel adventure to Old World Wisconsin. She'd been there before on a school outing and promised to show us the really good places. I could even take my camera if I promised to document our adventures.

One of the good places was the blacksmith's shop where, as the only kid, she got to do all the fun stuff like working the air pump to heat up the forge fire so the blacksmith could bend an iron bar into a hook. She  even got to buy S hooks like she help make to hang up her REALLY IMPORTANT stuff in her bedroom As promised I documented her every move with my D7000  set at ISO2000.

Then it was across the road to the German farmhouse where it was washing day. She got to help here too, doing everything from pounding a shirt in the rinsing tub to twisting one end as she and the old timer haus frau wrung it out. Again, as promised, I documented every step. Then, horror of horrors, I discovered I'd forgotten to reset the camera's ISO. All the pictures of this REALLY IMPORTANT part of the adventure were overexposed!!

Can they be rescued? Ask RawDigger. Like RawTherapee it's a free download found at  http://www.rawdigger.com/  It's easy to use and will tell you all the things about your raw photos that you never thought to ask.

RawTherapee's highlight reconstruction works by using the data in the less damaged RGBG channels to fix the data in the totally blown channels. In this RGB render mode, you are looking at a rough color image created without a true demosaic or white balance  You can also look at the composite mode which is very green or at the individual color channels.

The red splotches show the blown areas: in lighter red where there is data in at least one channel; in darker red where all data is blown out. RawDigger will also shows underexposed areas using blue splotches.

The program has another feature  worth the downloading time if you are ever faced with a directory of overexposed  images. With a CTRL + left or right key I quickly stepped through the overexposed NEF files to grade them from 'easy fix' to 'this will take serious work in GIMP or Photoshop.'

I graded this image as moderately difficult. While there are numerous bright red patches that will cause problems they are not in areas that would do major damage if they didn't fully clean up. So for this tutorial I'll concentrate on Charlotte's shoulder and an area in the closest tub.

Next question:  what highlight reconstruct method should I use?  The choices are:

Luminance Recovery

CIELab Blending

Blend

and Color Propagation

With this image Color Propagation is so obviously superior--the wood grain in the tub, the strap on the shoulder-- you might ask why are the other methods still on the menu. One problem is Color Propagation can produce a reddish color cast. Something that is fine for skin tones but not so fine for violet tinted skies. A second problem is it may not work as well with other camera brands.

It certain doesn't work with all camera profiles. This image would have been an 'easy fix' if it wasn't for the way Nikon compresses data to a supposedly lossless raw file. Except for a few speckles the blue channel is not blown. At the moment RT doesn't have a D60 input profile so I cleaned it up using the camera standard profile.

And this interesting monstrosity  is what happened when I switched profiles to the Nikon D60 Adobe Standard.

All profiles are not created equal.

Finally this is my rescue workflow.

Reduce the exposure. With this photo by about -2 EV.

Go to Highlight Reconstruction with Color Propagation and let RT do its magic.

Charlotte's face is too dark.  Go  to CIECAM02 and added a parametric Lightness curve

That's it, Not perfect but from what I saw with RawDigger what I expected  Once all color channels are blown the data is gone forever and not even RT can bring  back fine detail like the wood grain in the bucket.. But RT can reconstruct colors in areas like Charlotte's shoulder where there is no important detail,  Plus in the final rendering the transitions are not as sharp as they look in the view boxes.

Tell the truth. If you hadn't been following the tutorial  would you have noticed Charlotte's shoulder or the wood grain in the bucket in this final image.

I'd like to thank iliasG and DrSlony for the advice they gave me. While any mistakes here are mine, if they hadn't set me straight on a few key matters, lets just say things would have been embarrassing. Some of my assumptions were not, in political media jargon, 'true facts.'

edit- DrSlony, an RT developer, sent me his workflow;

Scribble I use the "Blend" highlight reconstruction mode by default. Most of my shots are landscapes, which means I usually have time to plan the shot and there are not unexpected over-exposures. The Blend mode makes the transitions from well-exposed to over-exposed smooth and it keeps the overexposed area white, which is usually what I want. Color Propagation is amazing at reconstructing overexposed skin, so whenever that's the case I use CP. CP is also the only method for which you can move the Highlight Recovery slider well past 100. When you do that with the other methods, the overexposed areas quickly get dirty grey or some other form of ugly, but with CP you can easily go over 200 or even 300 and get a great looking result. CP fails at large overexposed areas, and it's slower than the others.

Anyone else who wants to contribute his or hers experience with RT is welcome to join in.


You can find this build of RT at-
http://www.visualbakery.com/RawTherapee/Downloads.aspx
The online manual is at-
https://docs.google.com/document/d/1DHLb_6xNQsEInxiuU8pz1-sWNinnj09bpBUA4_Vl8w8/edit