Tone Mapping using CIECAM02

Edit--Preliminary results of the "Perfect Monitor Chase" (see next post)
Compared to flat screen monitors my Sun workstation monitor is too dark by about .5 ev. So what I thought was perfect exposure and color--what this blog post is about--will--most likely--look overexposed and washed out on your monitor.

Why 'most likely'?  Because flat screens are all over the place. My local library has a wide assortment. On Internet 7 this blog post looked pretty good. On Internet 2 its colors were so garish I cringed. So I leave this edit  by saying the examples I posted might look terrible but I still stand by the workflow instructions.

Plus, if anyone knows an accurate way to set up the gain and bias of the three RGB guns in this beast leave a comment to point me in the right direction.



In my last post on CIECAM02 I mentioned tone mapping  without going into details. While I did have a build, #181, that integrated both tools, my sys7 64 bit windows system did not like the build. It was compiled on a 32 bit Linux computer and worked great with that operating system.  But it came without a windows installer to register  its dlls and after an average of ten slider adjustments it crashed. Not an ideal way to learn how to use a new tool.

Edit  Build 185 is now available for both 32 and 64 bit PCs. Looks to be rock sold and faster than before.You can find it at Ollis's site:
  http://www.visualbakery.com/RawTherapee/Downloads.aspx


Despite the crashes, the build gave off hints of great things to come. So, without any great expectations of success, I went down to the old office turned junk room to dust off my old 32 bit Vista machine. To my surprise, as long as I didn't run out of memory during a conversion, the build worked.

I started with a snap of Matilda before we went to the bowling alley for Charlotte's birthday party. It was the first in a series so I suspect I took it to test the lighting conditions. Not a totally bad snap, but it needed fixing before I could email it to Matilda.

Without going into the details which I'll save for after a final release, here are the results.
The old way, using LAB mode and without CIECAM02.

And the new way

Before CIECAM02 I might have been satisfied with the first version, but there is no question that the second version does a much better job correcting the blown highlights in Matilda's hair.

Her face is still shaded more that I like with red channel number of about 50 percent. If I increased the strength of the tone mapping to lighten it, white blotches with a black center appear on Matilda's little hair pin and then on the blown parts of her hair. This is not a bug. The white happens when tone mapping runs out of data (RGB 255, 255, 255) and the black when the algorithm overflows. On PCs and perhaps MACs this produces a negative number, something RAWTherapee and other image processor interpret as black. (RGB 0,0,0).  Even RAWTherapee can't fix everything.

In my last post I also said I hadn't found much difference between the three 'lightness vs xxxx' algorithms. This time, with Matilda's hair, the 'lightness vs chroma' worked best. CIECAM02 also has its built in "lightness' curves. I used those to bring up the red channel on Matilda's forehead to about 62 percent.

Which brings up another complication. After I installed my #181 build on my old 32 bit machine I created the three tone mapped images. They looked good downstairs. But when I brought them up to my 64 bit machine they looked much too dark.  Last summer  I blogged about how accurately I calibrated the antique Sun Workstation monitor using paint chips but CRT monitors do drift, especially older ones. As for the monitor downstairs, it is a flat screen that I bought in a garage sale for $2-- one not guarantied to be on the gold standard for monitor accuracy.

My laptop and third computer is currently in parts since I have to replace its CPU fan, so I went over to my next door neighbor to check how the images looked on his new laptop--somewhere in between what I was seeing on my computers.

This is a problem that the old timer darkroom print folks in my computer club have been trying to pound into the heads of us new timer digital folks. You can not guarantee how light or dark the images you slaved over to get perfect will look when they pop up on other people's monitors out in Internet Land. So if you enjoy these tutorials  please take a moment to add a comment--just a 'too dark' or 'washed out' or 'about right'  to let me know what you are seeing.  Blogging on how great RAWTherapee is while using bad imagery isn't convincing.

Double Thanks.
scribble


EDIT
Sorry if you ran into the "prove you are not a robot" thingy during a comment. Now off. So comment away.

Noise Reduction RT vs 'the others'

Two months or so back I had this post all planned out  The 'other' was to be my antiquated version of Adobe Camera RAW that came with Photoshop CS3. When I compared the noise on two of my images RT clearly lost with one image but tied with the second. Slightly confusing.

So I checked the RT forum  to see if there was some info I had missed. There I learnt that the noise reduction routines were being  rewritten.  I decided to wait for the revision. Then life interfered with blogging and I didn't come back to this until a few day ago.

mbod had already done the comparison where his 'other' was NeatImage.  The link for the discussion is http://www.rawtherapee.com/forum/viewtopic.php?f=1&t=3708  He provided this screenshot; I measured the noise in the sky with ImageJ

The interesting thing about these measurements is the noise profiles are nearly identical even though the RT processed image on the right looks much noisier. (click to view original size) That's caused by the relatively large chromatic blotches emil talk about in the forum post. They are what you see but since a line profile only measures noise that is a pixel wide, it makes the Signal /Noise look better than it really is.

Normally I do most of my photographic work on my 32 bit desktop machine. While I have 32 bit memory management problems, I also have a semi ancient Sun workstation monitor that was built to last.

And has lasted. With a large 1600 pixel screen, the monitor can be calibrated and color managed so the colors and values on the screen are identical to those I see on my prints. Just as important when I raise or lower my head, the image intensities don't change like they do on my laptop screen.

But this time I used the 64 machine after I downloaded mbod's RAW file. And I was surprised by what I saw after I ran it through RAW therapee.

The noise profile of the 64 bit jpg (left) was much less noisy than I expected. More important it didn't have the chromatic blotches that I have in my 32bit jpg (right) and that mbod had in his conversion.  (click the screen captures to view their original size). Big shock to discover there was that much difference between the two versions of RT.

     Both jpgs were converted using identical settings. The formula used was exposure compensation to correct for underexposure followed by RL sharpening with a radius of .50 and amount of .24. This was followed  by impulse noise reduction of 75 and luminous noise reduction of 49. All other settings were left at the default values.

glascort also posted a NeatImage conversion which I compared to my 64bit jpg.  While my noise is 4 times lower I lost fine detail like the guy wire on the tower (circled in red).  Larger detail like that on the side of the building (circled in blue) wasn't affected. But there is no question that my version is less crisp than the glacort's version.

That turned out to be rather easy to fix. I loaded the 64bit jpg into RT and upped the Lab mode contrast to around 30. That brought out more noise so I added a bit more impulse noise reduction. As you can see in my reworked jpg on the left, the noise is still lower than the NeatImage jpg, the overall image is as crisp and I recovered the lost guy wire (circled in red).

   While I don't have any of the 'others' installed on my 64 bit machine yet, I feel the current 4.0.6.3  64 bit noise reduction is quite good. So we might be far closer to overtaking ACR than some believe.

A Golden Sky

Last Thursday in the ongoing pursuit of bargains, I took a break from more serious errands after I spotted a garage sale sign. A good number of blocks later without passing a garage sale I was about to turn around.  Then I spotted a second sign. Since it was mid afternoon of a blustery rainy day the sign was half blown over and not immediately readable from the car.  But since its road only went one way I turned and headed down the side street.

This time I drove several winding looping blocks without a hint of a garage sale.  Then I came on an entrance to one of the city's many neighborhood parks. For the moment it wasn't raining and my D7000 kit was on the back seat so I stopped to explore any photo opportunities. And found this addition to the photographic genre, atmospheric skies.

Yes, for the zillionth time I forgot to switch the white balance to auto after I finished shooting indoors under compact fluorescent lighting.

But that is what the white balance dropper is for. I clicked in the circled area.

And ended up with this image

Sort of what I was after--an essentially monochrome RGB image. But it lacked that extra punch.  I decided to experiment and see how well RT could tone this image.

The HSV flat field editor in the color tab had worked well on other more colorful images.  A disappointment here. With so little color to work on, I saw only a slight shift towand red.

I went back to my tried and true toning procedure-LAB mode in the exposure tab. As I've blogged about earlier the b color channel controls the ratio of the two complimentary colors, blue and yellow. Took only two simple drags to adjust their ratio and few more slider adjustments to fine tune the image.

A couple builds back I would have stopped here  but there was one more routine to try, the channel mixer.

Because of bugs it hadn't worked as expected the last time I tried it on some infrared images. But according to the RT forum, all those bugs had been swatted in time for the latest 4.0.3.4 build.

Nobody fibbed.  I moved the slider around without any particular plan or science until I found a richer color than I had using the LAB mode.

The  final image.

So download RT and try it on your images. You won't be disappointed.

PS. On my way back, I passed the garage sale a couple blocks farther down from where I made the wrong turn. Bought a neat junk/ treasure that will be used-a heated coffee carafe complete with a thermostatic control that varies the temperature from just lukewarm to burn you tongue.

Now, while I drive on a photo day trip, my coffee fiend of a wife can sip her favorite brew as she scans to countryside for more photo opportunities.

Serendipity,  wouldn't you say.

Sharpening with RAW Therapee-- Part 1

Let's start way back at the basics. What makes up the pic of your dog that you just brought up on your computer screen?  A very long list of numbers.

This is a small sample of the 5,161,117 numbers that make up my DSC8706 jpg   It's at the boundary between the header--the zeros at the top-- and the numbers that tells a computer how bright or dark to set the three colored dots that make up the pixels on its computer screen--the hexadecimal numbers at the bottom.

At the top of the header is the EXIF data--the list  of camera settings. Following that is more data about the lens I used.  Finally there are pages and pages of zeros that I could fill up with metadata. If I looked up the ascii numbers for 'This is going to be tedious!!' and individually replaced some of the zeros that sentence would show up in the metadata. If I still hadn't used up all my tedium quota for the day and replaced the picture data with zeros I would eventually draw a black line across the top of  DSC8706.

Once upon an ancient time that was how data editing was done.  And shortly after that ancient time, when monitors that could draw pictures appeared beside humongous CPU units, programmers began writing image editors to take away the tedium of doing such mathematics on these lists of numbers.
 
Sometimes we only want to modify a pixel or two--covering up hot pixel noise--but usually we want to do mathematical operations on all or a good part of our image.  For example, to increase the effective exposure by a stop we multiply the image numbers by 2. Or to decrease it by a stop, we multiply by 0.5. Or we multiply them by a range of numbers that depends on how far and in what direction we drag a point on a tone curve.

To sharpen an image with an unsharp mask we do "a simple linear image operation—a convolution by a kernel that is the Dirac_delta minus a Gaussian blur kernel."  As per  http://en.wikipedia.org/wiki/Unsharp_masking

All clear and understandable?  Right?  Or did you party big time the night before that early morning math class and  could use a refresher explanation? Maybe you even forgot how you sharpen an image with, of all things, an unsharp mask.

Unsharp masking goes back to a time when you would stick your head under a big black cloth and take your image on a large single sheets of film.  Once you developed  your image of a black bird flying up in the light blue sky, you would have the B&W negative, a white bird in a black sky. If you projected this on another sheet of film, after development you would be back to a positive, a black bird against a lighter sky. If you did this as you would with a print, well focused, you would now have a  'sharp' mask.

Which wouldn't be much use. Instead. if you defocused your enlarger you would have a blurry and wider positive of your black bird, your 'unsharp' mask.  When you carefully aligned the original negative and its mask and put the stack in your enlarger you have a halo around the edges of the black bird. This combination will make a print with more edge contrast than a print made with just the original negative. There isn't more detail but because of the way our eye/brain system works this print will look sharper.

Back in those film days this was another  'great in theory' procedure that hardly anyone ever used.  It wasn't until our digital days when all you had to do was move a few sliders to  ' convolute them sharpening kernels' that it became part of a photographer's digital development work flow.

Since a quick google of  'unsharp mask tutorial' will show why the world doesn't need another unsharp mask tutorial, I will talk about what happens when I go out my way to do it wrong.

Instead of a blackbird I start with a seagull flying over a field and pond--a peaceful and quiet country scene. (Which it ain't. The second largest mall and one of the busiest intersections in the city was behind me.  A tidbit of info I'll toss in to point out that in photography framing may not be everything but it is still is a good hunk of everything.)

I ran the unsharp mask sliders up as far as RT would let me before I decided that image would be too ugly even for this do-it-wrong tutorial. So I pulled them back slightly.

Usually I see white halos when I over-sharpen, but that is because nature gave us far more darkish subjects than pure white subjects to photograph. The bird's white head has a well defined black halo while its black feet has a more traditional white halo. In the third window where there isn't much contrast between the upper part of the inner wing and the soccer field the halo is barely visible.

Traditional versions of unsharp mask have only three sliders--radius, amount and threshold.  RT's version has five--Sharpen only edges and  Halo control. Sharpen only edges work to prevent the sharpening of noise noise pixels. I'll go into that in another tutorial.

Activating halo control didn't completely eliminate the halos, I'd gone too far with the other settings, but it did reduce them considerable. With more normal setting, a radius of 0.8, an amount of about 150 and the default threshold of 512 I doubt I would have had to use halo control.

In the end how did I sharpening the image? By using only the contrast by details routine, a feature not found in other RAW (or jpg) converters.

What this routine does is use some fancy math to select areas that have details. The areas range from 1 or 2 pixels wide (Finest) up to about 16 to 20 pixels wide (Coarsest).  Then RT multiplies those areas by whatever number you select to make them stand out in the image. Or stand out less in the image if you move the slider towards zero. With a high ISO images, one to two pixel areas are almost always noise spikes. Then I set the Finest slider down to around zero.

As with all sharpening routines it is easy to get carried away. With flower macro images setting the numbers 1 and 2 slider to slightly under 2.0 works well.  With this image which has motions blur I went a bit farther in what I must confess is a rescue operation.

I was driving out of the parking lot of my bank when I spotted the seagull.  I had my camera in the back seat but didn't have the time (or forgot) to set the camera up for bird shots. I snapped this image at 1/500 sec instead of a motion stopping 1/1200 sec. While RT can and has hid many of my photographic mistakes, with motion blur it can't perform miracles.

And don't be surprised if RT slows down. It's one of RT's most CPU demanding routines. With 16.2 mp images and using a less than a speed demon, four year old, 32 bit machine it add a minute or so to the RAW conversion  But you will find the extra time worth it.

The top window is with sharpening, the bottom is without.

So, grab your camera and best birding lens and go off for the image that will wow the world.  RT has all the power you need.  The rest is up to you.

RAW Therapee's Batch Processing to the Rescue

Autumn, a long time flickr contact, had a problem. During a vacation trip to Florida two years ago she went sailing and took a pile of photos with her Coolpic 550s P&S. Unfortunately her camera was accidentally setup with an EV of -1.7 and all her jpg pics came out ~4X underexposed.

Batch processing is not a new RT feature but until recently I hadn't realized how powerful and useful it could be.  I've taken my share of images with a setting wrong--white balance, exposure, etc--before I noticed and corrected it. So here is a mini tutorial using some of Autumn's photos on how I would use RT to hide these mistakes.

Start RT in its normal multi-tab mode and open the folder with the underexposed photos. Start by upping the exposure  +1.7 in one of the photo using the Exp Comp slider. Name and save your new profile.

Go into the file browser and control click on the other underexposed images.

Right click on the selected photos. This will bring up the new batch interface. Click on Profile Operations, Apply Profile, and in this case Autumn Underexposed, the name of profile I just saved.  Your copy of RT has a profile folder that holds all the default profiles that come with RT plus any you created and saved.

After a few seconds the thumbnails will brighten up. What have you done? Like Adobe Lightroom you haven't changed any of your photos. Instead you have created a pp3 file for the photos you selected which tells RT what changes you want to make.

What's next? If exposure correction was all you wanted to do, you could hit ctrl Q to put them into the Batch Queue. After selected what folder you want to save them to and the desired file format--jpg, 8 and 16 bit Tiff or PNG--you can hit start processing and let RT do its thing.

I preferred to fine tune the three photos I was working with.

The horizon was tilted, always happens with rocking boat photos, so I corrected that with the straighting tool  The overall photo was now exposed correctly but Autumn and her sister Jordan--wild hair--were still too dark. I used the LAB mode luminous channel on the Exposure tab to create a tone curve. I could have done this with the more common RGB mode but I think the LAB mode is cleaner since it doesn't introduce color shifts. Beside LAB mode editing on a RAW file is one of many features unique to RT. I also noticed that parts of the T-shirt were blown out so I use the Highlight Recovery slider to correct that problem.

After I finished editing the photos I put them into the batch queue.  There are two versions of the air plane image, each with different editing. Once I convert them I'll view them full size and decide which one I prefer. It's a great timesaving feature and a few days ago I converted the same photo 14 ways in one operation to check out the new profiles that showed up in the latest build of RT.

Although it wasn't necessary since I have 4GB of memory on this computer. I also closed everything but the batch queue and file browser.  If you look close, I forgot to circle it, you can see where the memory usage dropped in the Windows Task Master window.  RT can do many great things very well but that requires more RAM memory than the less useful free (and expensive commercial) editors. So if you have an older computer watch your memory usages.

How many great things can RT do very well? Here is the current list

If you click on Apply Profile (partial) you can pick what features you want to use from your saved profile. It is also a great overview of what RT can do

Finally I must thank Autumn for letting me use her photos in this tutorial.  You can find her flickr photostream at  http://www.flickr.com/photos/syksylla/