Rescuing REALLY IMPORTANT snapshots from overexposure

Last summer Charlotte took her mother, her grandmother and me on a time travel adventure to Old World Wisconsin. She'd been there before on a school outing and promised to show us the really good places. I could even take my camera if I promised to document our adventures.

One of the good places was the blacksmith's shop where, as the only kid, she got to do all the fun stuff like working the air pump to heat up the forge fire so the blacksmith could bend an iron bar into a hook. She  even got to buy S hooks like she help make to hang up her REALLY IMPORTANT stuff in her bedroom As promised I documented her every move with my D7000  set at ISO2000.

Then it was across the road to the German farmhouse where it was washing day. She got to help here too, doing everything from pounding a shirt in the rinsing tub to twisting one end as she and the old timer haus frau wrung it out. Again, as promised, I documented every step. Then, horror of horrors, I discovered I'd forgotten to reset the camera's ISO. All the pictures of this REALLY IMPORTANT part of the adventure were overexposed!!

Can they be rescued? Ask RawDigger. Like RawTherapee it's a free download found at  http://www.rawdigger.com/  It's easy to use and will tell you all the things about your raw photos that you never thought to ask.

RawTherapee's highlight reconstruction works by using the data in the less damaged RGBG channels to fix the data in the totally blown channels. In this RGB render mode, you are looking at a rough color image created without a true demosaic or white balance  You can also look at the composite mode which is very green or at the individual color channels.

The red splotches show the blown areas: in lighter red where there is data in at least one channel; in darker red where all data is blown out. RawDigger will also shows underexposed areas using blue splotches.

The program has another feature  worth the downloading time if you are ever faced with a directory of overexposed  images. With a CTRL + left or right key I quickly stepped through the overexposed NEF files to grade them from 'easy fix' to 'this will take serious work in GIMP or Photoshop.'

I graded this image as moderately difficult. While there are numerous bright red patches that will cause problems they are not in areas that would do major damage if they didn't fully clean up. So for this tutorial I'll concentrate on Charlotte's shoulder and an area in the closest tub.

Next question:  what highlight reconstruct method should I use?  The choices are:

Luminance Recovery

CIELab Blending

Blend

and Color Propagation

With this image Color Propagation is so obviously superior--the wood grain in the tub, the strap on the shoulder-- you might ask why are the other methods still on the menu. One problem is Color Propagation can produce a reddish color cast. Something that is fine for skin tones but not so fine for violet tinted skies. A second problem is it may not work as well with other camera brands.

It certain doesn't work with all camera profiles. This image would have been an 'easy fix' if it wasn't for the way Nikon compresses data to a supposedly lossless raw file. Except for a few speckles the blue channel is not blown. At the moment RT doesn't have a D60 input profile so I cleaned it up using the camera standard profile.

And this interesting monstrosity  is what happened when I switched profiles to the Nikon D60 Adobe Standard.

All profiles are not created equal.

Finally this is my rescue workflow.

Reduce the exposure. With this photo by about -2 EV.

Go to Highlight Reconstruction with Color Propagation and let RT do its magic.

Charlotte's face is too dark.  Go  to CIECAM02 and added a parametric Lightness curve

That's it, Not perfect but from what I saw with RawDigger what I expected  Once all color channels are blown the data is gone forever and not even RT can bring  back fine detail like the wood grain in the bucket.. But RT can reconstruct colors in areas like Charlotte's shoulder where there is no important detail,  Plus in the final rendering the transitions are not as sharp as they look in the view boxes.

Tell the truth. If you hadn't been following the tutorial  would you have noticed Charlotte's shoulder or the wood grain in the bucket in this final image.

I'd like to thank iliasG and DrSlony for the advice they gave me. While any mistakes here are mine, if they hadn't set me straight on a few key matters, lets just say things would have been embarrassing. Some of my assumptions were not, in political media jargon, 'true facts.'

edit- DrSlony, an RT developer, sent me his workflow;

Scribble I use the "Blend" highlight reconstruction mode by default. Most of my shots are landscapes, which means I usually have time to plan the shot and there are not unexpected over-exposures. The Blend mode makes the transitions from well-exposed to over-exposed smooth and it keeps the overexposed area white, which is usually what I want. Color Propagation is amazing at reconstructing overexposed skin, so whenever that's the case I use CP. CP is also the only method for which you can move the Highlight Recovery slider well past 100. When you do that with the other methods, the overexposed areas quickly get dirty grey or some other form of ugly, but with CP you can easily go over 200 or even 300 and get a great looking result. CP fails at large overexposed areas, and it's slower than the others.

Anyone else who wants to contribute his or hers experience with RT is welcome to join in.


You can find this build of RT at-
http://www.visualbakery.com/RawTherapee/Downloads.aspx
The online manual is at-
https://docs.google.com/document/d/1DHLb_6xNQsEInxiuU8pz1-sWNinnj09bpBUA4_Vl8w8/edit

How Good is My Ancient Sun Workstation Monitor?

In my last post I mentioned an unexpected  advantage of upgrading my desktop--the discovery that  I now had a video driver specifically for my Sun workstation monitor.  Major result--my monitor colors looked brighter and more vivid.  Minor result--a software switch between my manual screen layout--squished in with black borders so the aspect ratio of  a square image is exact--and the driver's more open layout that fills the whole screen.

I began to wonder if I could go from 'this looks better'  and come up with some tests where I could say 'this is exactly how much better" the colors appeared.  Something like I'd attempted after I bought my first digital camera and wanted to reverse engineer how digital cameras worked. Then I photographed a fan of paint cards and compared the swatches to the colors I saw on my monitor.  With mixed results if I remember correctly.

A decade later I now understand why my naive first attempt at color management ended a bit off.  Somewhere in the house is an extra thick tome on the complexities of  eyeball, brain, camera, scanner, printer, press and monitor color processing.  Someday I might even finish reading it. During my last attempt my brain seized up when I hit the complexities of mating CMYK processing with the ink spread on various paper grades during high speed offset printing.

But I did manage to retain a few key facts. The first is color calibrate your monitor. If that is off everything that follow will be meaningless. To correct gross problems you can fiddle with your monitor controls  and attempt to match images you find on the web. But to calibrate accurately you have to buy hardware.

I invested in a Pantone Huey--at the time $99 at my local we-stock-everything craft store.  The Huey has its quirks.  The software in the box dated from the Win 2000 era and did not install on my newer OS.  Once I located the upgrade, more of a hassle to find than it should have been, I couldn't make the 'adjust the contrast and brightness control so the top right circle... etc etc' part of the setup work. In the end I set them to 75% and clicked next. Despite these quirks, the Huey did cleaned my obvious color management problems. Still I never did discover how accurate the color matches were

So I was thinking of getting out the camera and repeating the paint card experiment when I discovered http://www.color-swatches.com .  Which, not surprising, is a site with a data base of paint swatches. They even provide the RGB numbers used to create the swatch and CMYK numbers to print the color cards. Below is an example of what you find there

So all I had to do was collect some paint card.  Unfortunately finding them in stores I happened to pass while doing grocery shopping didn't work out well. Walmart's ColorPlace wasn't in the data base. Home Depot's Glidden had only one swatch on line that matched the cards I took.  In the end I made a trip to Sherwin Williams. It has a swatch for every paint it sells. This combined with larger than normal paint cards printed on thicker than normal paper stock makes it my recommended store for swiping your cards. 

With cards in hand I download a swatch.  And I hit my first technical problem.  Monitors blasts out light. The paint cards, on the other hand, reflect light. Depending on the light quality (intensity, color temp and all that jazz) they will, more often than not, look different than the monitor swatch.  After holding card by the monitor I could say the swatches sort of looked the same. but that wasn't the accurate one-to-one comparison I wanted.

After some fiddling I discovered if I opened the curtains of a window to the left of monitor and angled the card up or down to catch more or less light I could match the intensities.  But I still ended up with a so-so color match.  Close but hardly perfect like in the photo below.

I must confess to spending far too much time fiddling with controls while the answer to the problem stared me in the face.  Quick question. What is causing the green cast on the swatches of the lighter blue paints?  After all Sherwin Williams wouldn't mix a little green pigment in with its expensive blue paints.

All I had to do was carry the card to the window, twist it first to face the window light and then away, to watch it switch between slightly greenish and pure blue. The Wisconsin clear sky sunlight had started out pure. But once it was reflected off the green leaves that shade the patio below the window it had a greenish component.

Problem solved. Color cast in the light.  Factor that out and the match between real object and monitor image colors becomes almost perfect.  Some days life is good.

Or close to good. Measured with Gimp, the RGB numbers on a swatch are what www.color-swatch.com says they should be.  Measure with RawTherapee using its neutral profile, one that is supposed to display an image exactly as shot, and the numbers end up 5 to 9 units lower  So after I post this I should be a good RT user and also post a bug report.

Edit

The  RGB 5-9 off bug is when I save a swash snip in png format. When saved in jpg format the numbers are off by 1.

And not surprising when you think about it, the green colorcast that had me twirling about mentally yesterday depends on the time of day and where the sun is in the sky.  Around noon when trees are in direct sunlight the reflection off the leaves is a problem.  But earlier this morning when I repeated the test and the sun was much lower in the sky the match was so close I could hardly make out the seam where the card touched the monitor swatch.

Ain't color management tricky ... but fun.