Slider Shock or What Do I Do Now

You have  RT on the screen in front of you. You have read the first pages of the manual and now have the file browsers loaded with a directory full of images. You have selected your first RAW file. You are ready to fix problems.  Your snapshot will become a sent-to-all-the-grandmas minor masterpiece.

Then it hits you. Big Time.  Slider Shock!!. You think "Never Never in a Zillion years will I Learn how to use All those Sliders and Check Boxes."!! You mutter "I want my money back"! Then you remember RawTherapee is free.

Happen to everyone. Even me. And back then there were a bunch less sliders. But by now I've discovered the big RT secret.  Learning how to use all the sliders is for hang-on-the-wall-and-be-awarded-the-ribbon masterpieces. For the send-to-grandma ones you only use a few sliders. As for the rest--I must confess it wasn't until my last post that I needed to know much of anything about what was going on inside the RAW section. You can pick up most things as you go along.

This is a common exposure problem. Too light outside the window. Too dark inside the kitchen, The only thing correctly exposed  is the flowers on the window sill.  

The send-to-grandma workflow in seven easy steps:

Exposure compensation to 1.00.  This lightens Charlotte-the grandma interesting subject--but blows out the outside  foliage.

Enable HighLight Reconstruction. Because we are working with sky peaking through foliage I used the blend mode. The highlight recovery sliders-out of the picture- are at their default settings.

Now we are dealing with what I am beginning to think of as the twin power tools of RT. The four entries in  the history box appears when you set up Tone Mapping and CIECAM02 to work together. Just follow the steps in the Tone Mapping tool tip.

I used the control cage version of the Brightness curve and dragged it up until the snap was brighter than I wanted.

I fine tuned the snap with the Brightness and Colorfulness sliders. This shows the setting I decided to go with.

Rather than loading the snap into the queue. I copied my new procedure to the clipboard.

Finally I went back to the file browser, selected raw files with similar exposure problems, pasted my procedure and sent them all to the queue to be turned into grandma happy jpgs ready to be printed or emailed.

A couple of the jpgs that came out of the queue.

Not hard at all,  If you want to do additional work --a crop for instance--load up the raw files again. They will come up in 'last saved' mode ready to be worked on.

Rescuing REALLY IMPORTANT snapshots from overexposure

Last summer Charlotte took her mother, her grandmother and me on a time travel adventure to Old World Wisconsin. She'd been there before on a school outing and promised to show us the really good places. I could even take my camera if I promised to document our adventures.

One of the good places was the blacksmith's shop where, as the only kid, she got to do all the fun stuff like working the air pump to heat up the forge fire so the blacksmith could bend an iron bar into a hook. She  even got to buy S hooks like she help make to hang up her REALLY IMPORTANT stuff in her bedroom As promised I documented her every move with my D7000  set at ISO2000.

Then it was across the road to the German farmhouse where it was washing day. She got to help here too, doing everything from pounding a shirt in the rinsing tub to twisting one end as she and the old timer haus frau wrung it out. Again, as promised, I documented every step. Then, horror of horrors, I discovered I'd forgotten to reset the camera's ISO. All the pictures of this REALLY IMPORTANT part of the adventure were overexposed!!

Can they be rescued? Ask RawDigger. Like RawTherapee it's a free download found at  http://www.rawdigger.com/  It's easy to use and will tell you all the things about your raw photos that you never thought to ask.

RawTherapee's highlight reconstruction works by using the data in the less damaged RGBG channels to fix the data in the totally blown channels. In this RGB render mode, you are looking at a rough color image created without a true demosaic or white balance  You can also look at the composite mode which is very green or at the individual color channels.

The red splotches show the blown areas: in lighter red where there is data in at least one channel; in darker red where all data is blown out. RawDigger will also shows underexposed areas using blue splotches.

The program has another feature  worth the downloading time if you are ever faced with a directory of overexposed  images. With a CTRL + left or right key I quickly stepped through the overexposed NEF files to grade them from 'easy fix' to 'this will take serious work in GIMP or Photoshop.'

I graded this image as moderately difficult. While there are numerous bright red patches that will cause problems they are not in areas that would do major damage if they didn't fully clean up. So for this tutorial I'll concentrate on Charlotte's shoulder and an area in the closest tub.

Next question:  what highlight reconstruct method should I use?  The choices are:

Luminance Recovery

CIELab Blending

Blend

and Color Propagation

With this image Color Propagation is so obviously superior--the wood grain in the tub, the strap on the shoulder-- you might ask why are the other methods still on the menu. One problem is Color Propagation can produce a reddish color cast. Something that is fine for skin tones but not so fine for violet tinted skies. A second problem is it may not work as well with other camera brands.

It certain doesn't work with all camera profiles. This image would have been an 'easy fix' if it wasn't for the way Nikon compresses data to a supposedly lossless raw file. Except for a few speckles the blue channel is not blown. At the moment RT doesn't have a D60 input profile so I cleaned it up using the camera standard profile.

And this interesting monstrosity  is what happened when I switched profiles to the Nikon D60 Adobe Standard.

All profiles are not created equal.

Finally this is my rescue workflow.

Reduce the exposure. With this photo by about -2 EV.

Go to Highlight Reconstruction with Color Propagation and let RT do its magic.

Charlotte's face is too dark.  Go  to CIECAM02 and added a parametric Lightness curve

That's it, Not perfect but from what I saw with RawDigger what I expected  Once all color channels are blown the data is gone forever and not even RT can bring  back fine detail like the wood grain in the bucket.. But RT can reconstruct colors in areas like Charlotte's shoulder where there is no important detail,  Plus in the final rendering the transitions are not as sharp as they look in the view boxes.

Tell the truth. If you hadn't been following the tutorial  would you have noticed Charlotte's shoulder or the wood grain in the bucket in this final image.

I'd like to thank iliasG and DrSlony for the advice they gave me. While any mistakes here are mine, if they hadn't set me straight on a few key matters, lets just say things would have been embarrassing. Some of my assumptions were not, in political media jargon, 'true facts.'

edit- DrSlony, an RT developer, sent me his workflow;

Scribble I use the "Blend" highlight reconstruction mode by default. Most of my shots are landscapes, which means I usually have time to plan the shot and there are not unexpected over-exposures. The Blend mode makes the transitions from well-exposed to over-exposed smooth and it keeps the overexposed area white, which is usually what I want. Color Propagation is amazing at reconstructing overexposed skin, so whenever that's the case I use CP. CP is also the only method for which you can move the Highlight Recovery slider well past 100. When you do that with the other methods, the overexposed areas quickly get dirty grey or some other form of ugly, but with CP you can easily go over 200 or even 300 and get a great looking result. CP fails at large overexposed areas, and it's slower than the others.

Anyone else who wants to contribute his or hers experience with RT is welcome to join in.


You can find this build of RT at-
http://www.visualbakery.com/RawTherapee/Downloads.aspx
The online manual is at-
https://docs.google.com/document/d/1DHLb_6xNQsEInxiuU8pz1-sWNinnj09bpBUA4_Vl8w8/edit

In search of the perfect eyeball, monitor, camera, printer calibration

My search for perfect color calibration began during a service call from the charter.com folks.  Even thought the signals bars were strong and the cable box had been changed the TV card on this computer was missing channels. Turned out that by some quirk of the transmission, reflection and absorption properties of cable connections, the frequencies of my missing channel were being dragged into kaput-land. Moving heavy furniture and replacing a cable brought them back to life. Turns out cable TV electronic is as quirky and finicky as color management.

During the visit I resurrected my old 32bit machine to avoid running up and down two flights of stairs. It now  has a small flat screen monitor and  I saw that my carefully adjusted image of Matilda (  http://scribble-jpc.blogspot.com/2013/01/tone-mapping-using-ciecam02.html  ) looked too bright.  A trip next door to look on my neighbor's laptop confirmed my big OOPS. What I thought was an accurately calibrated monitor, my Sun workstation 567, was running a half stop too dark

To avoid another big OOPS I set my goals. I would recalibrate using my collection of Sherman Williams paint swatches. (  http://scribble-jpc.blogspot.com/2012/05/how-good-is-my-ancient-sun-workstation.html  )  From http://www.color-swatches.com I would download virtual swatches that came with HSL-- Hue, Saturation and Lightness--and RGB numbers. I would adjust, adjust and adjust until I could lay the real swatch of the paint card against the virtual swatch visible on the monitor and not see a difference. Finally I snap a photo of that comparison with my D7000, display the image, and compare virtual to real again.


That turned out to be a quirky, tricky task. Not surprising since you can buy 500 page books just on the joys and groans of photographic color management.

I  use Sherman Williams paint cards primarily because they supplied color-swatch.com with data for all the paints they sell. Other paint vendors don't. They also print good quality paint cards. Cards from other paint vendors will obviously work but for the record the photo above shows red #82 , green #105, blue #116 and yellow #86. If I was going to repeat this I would switch out the yellow for a card with a wider  range of colors.

My Sun567 monitor has dual color adjustment, the usual brightness and contrast plus gain and bias adjustments for the individual color guns. I won't go into details about that leg of my search since it would be highly specific to a now very rare monitor except to say my attempt to use my D7000 spot meter as a radiometer fell apart because of the Sun's CRT raster scanning. In the end I ran the color gun gains up and adjust the monitor brightness by eyeball. Anyway, if you are using a typical flat screen monitor and not the more expensive professional models you don't have these adjustments. You take what the manufacture gives you.

To photograph and to view the paint swatches  I bought two ecosmart  120 watt equivalent  1100 lumens 5000k daylight flood lamps from Home Depot. I set the white point of Sun567 to 5000k instead of the more normal 6500k to match the lighting. For the D7000 I shot at the neutral jpg setting using a custom WB taken with a 18% grey card.

The  red circle shows the virtual swatch vs the paint chip and measures monitor calibration. The blue circle shows the double  comparison. Both jps's are straight out of the camera without any manipulation and displayed using IRfanview.

I was pleased with the results as long as I ignored the swatches above and to a lesser extent below the matches. These comparisons went bad fast. But there are technical reasons for the problem.

The two swatches that match are titled 'picnic' and 'organic green'.  Their Hue, Saturation  and  Lightness values are  108, 37, 69 and 105, 31, 79.  These values are created by Photoshop in the virtual world and by laying down inks on card stock in the real world. They are fixed.

In the photography world the numbers are not fixed. Since we are dealing with subtractive color Hue Saturation and Lightness are derived from the spectra and intensity of the light being reflected off the paint chips, Something that is far from constant.  And something sensitive to small differences.

How sensitive? If you look closely below the circled areas you can see what looks like a small dirt spec. It isn't. I'm holding the paint card. I can't see any dirt  or change in color.  But  I can feel a slight dimple in the card stock. Since the spotlight is above and slightly in front of the monitor that dimple changed the incident light reflected back into the camera enough to darken the area and cause a hue shift.

The triangle thing in the upper right corner is the sheet of paper I moved about with one hand to attenuate the light as I snapped photos with my other hand. I took more than a few before I managed to adjust the angle of the card, the intensity of light and even where I was sitting in front of the monitor. Taking a good image for this blog wasn't a 'snap and go' task.

Taking a good  'eyeball shot'  was easier. Twisting and bending the card until the colors blended and the seam between the monitor and paint card disappeared. This is also a good time to say that my eyeball matches were always better than my camera matches. The camera may be slightly off. Or maybe it is right on and my eyeballs are off. The CIECAM02 that I've been blogging  is about the quirky way we humans view the world.

I would have ended the post here if I hadn't googled for a bit more info on color management in the middle of writing it.  My Pantone huey calibrator came with a minimum manual, a single sheet, quick start guide. Since the Huey software has on screen instruction and defaulted to 'photo editing and web browsing' I hadn't worried much about the lack until I discovered that Pantone now has a pro version of the software. Its online manual told me I should use the 'Special Warm Medium Contrast' setting to match the 5000K lighting and monitor settings. Not a massive OPPS. All I had to do was change the setting, watch the colors shift and lay my paint card against the monitor. And... not quite a EUREKA moment...but close.

Matching four swatches rather than one or two is a significant improvement. So I won't  talk about how much time I wasted trying to work out a lighting set up so I could match all seven swatches in one photo.

Hitting my goal this close is the good news. Now the bad. My local library branch has a hodge-podge of computers equipped with monitors of different ages and aspect ratios. I checked out all the ones that weren't being used without coming close to matching the paint cards. Worse, one was off so much the lighter swatches were blown out.  The neighborhood center next door has newer monitors and better ambient lighting. My cards still didn't match, but the tonality and contrast looked more reasonable.

So while I have excellent color management on my system. I still publish most of my work on line. With so many uncalibrated monitors out in the world was all this effort worth the trouble? Comment are welcome.

Edit 3/1/13
The original post covered the eyeball, monitor, camera part of the calibration. Now the printer part.
Now and then I do a 4x6 on my old Epson, but the big prints I send out. This one was done at my local supermarket where they combine the best print job in the city with being the least expensive. They keep their Fuji Frontier 570 so well calibrated I see a near perfect color match with RawTherapee's sRGB output profile.

Now it's time to frame the print so it can go up on the wall tomorrow for my camera club's member show. Things are sweet.