Build 269-A final, for a couple months, wavelet tool

In earlier tutorials I've worked with raw files I've taken. Now, for several reasons including some consistent criticism that I haven't written the book that explains the intricacies of every slider and curve in tools as complicated as RT's wavelets, my tutorials will use raws and jpgs available on the internet. Then anyone can duplicate my setting and if they are interested in the other settings...go for it. Just come back and tell us if you discover something interesting.

My source is http://www.photographyblog.com/reviews/   They have a very large collection of raws, often with matching jpgs, from a multitude of cameras. The images I used for this tutorial shows a family group at the end of a subway corridor that are almost lost in the darkness and ISO128000 noise. A challenging shot that will showcase several RT tools.

The link for the jpg--we will compare the RT results with the straight out of the camera results--is
http://img.photographyblog.com/reviews/panasonic_lumix_dmc_g7/sample_images/panasonic_lumix_dmc_g7_36.jpg  The matching raw file is panasonic_lumix_dmc_g7_36.rw2

If you haven't upgraded to the windows build RT 4.2.269 or mac RT 4.2.270 do so immediately. If you are working with wavelets any earlier build will disappoint. Build 269 is stable (no crashes in about a week) and has several new features and bug fixes (a beta release for RT 4.3 maybe) plus some  new documentation.

Jacques, the chief developer behind wavelets and CIECAM has posted an updated RawPedia article in french. I'm sure is makes great reading if you are fluent in technical french, but after going through google translates  the meaning has been mangled in some sections. Example, when you come to a confusing sentence about a 'beach'  the correct translation is 'range' as in range of values. But for the most part it is  understandable and in this tutorial I will try to expand on what you will learn there

Wavelets are 3 dimensional. This mathematical third dimension is one of the reasons you see three sliders in many sub tools; two of which will confuse you because they seem to do nothing or make the image worse,  Some reasons for this behavior are:
1- they work much better with a different type of image
2-they are in the tool because Jacques and team have future surprises planned
3-they are  there because Jacques and team hopes we users will come up with some of the new surprises.

So if you hit a slider or curve that confuses you leave it at the defaults. They have been carefully chosen to work with the majority of images.

Enough talk, the workflow: Lets see what we can do with this  image

This is the matching jpg to the raw file we will use,  ISO 12800 and SOOTC, The goal is to create as decent an image as possible of the family group that are barely visible at the end of the subway tunnel without completely destroying the detail by the handrail  in the foreground

*****

I started with the the residual image tool. It got its name because it was what was left over after the calculations extracted the wavelets. What you do to the wavelets doesn't change it. And vice versa. You must recombine them either on your RT screen in real time in a 100% view or after a trip to the queue. That's one disadvantage when developing wavelet workflows. To chase down all the smal artifacts I develop test jpgs at key steps so I can look at them with other programs. Such as imageJ, a widely used and free NIH scientific image analysis program.

You can modify the residual image with non wavelet tools.  The shadow and highlight sliders lighten or darken their areas of the images. Their matching threshold sliders decides how large an area they effect. In past tutorials I recommenced  using CIECAM for contrast and chroma because of the tool's curves. Now with the new final tuneup curves ...you pays your  penny and takes your pick.

With this image  the shadow slider brightens the family group so I can see how bad the noise is.

*****

It is ISO12800 bad, clearly what Jacques calls high noise in his rawPedia article. I'm pleased that the noise reduction workflow I touched on in my Lorde tutorial  is now his recommended workflow. In the program flow the preliminary noise reduction in the second tab is now before the color conversion step. The wavelet denoise is now at the end and is intended to clean up any additional noise and artifacts created by sharpening and other adjustments.

*****

For the noise preclean I chose Lum 70 and Detail 30. Depending on the final results I can fine tune these numbers.

*****

To lighten the back area more I tone mapped the image. While wavelet tone mapping is less aggressive and more artifact free than RT's earlier version when it is combined with its gamma slider it can lighten and darken to both extremes.

It's worth mentioning that tone mapping works on the signal side of the all important signal to noise ratio just as noise reduction works on the noise side.  RT tone mapping uncovers noise in dark areas. It doesn't create it.  We are prone to saying the new full frame camera we bough to collect more light the light in its bigger pixels makes less noisy images,  Instead, when the exposure collects four times the light-- four times greater signal-- that is combined with twice the noise to double the signal to noise ratio. This may sound like super geek nit picking but I've seen forum discussions go off the rails into making wrong claims when people didn't understand or take into account the difference. A recent claim I've seen that tone curve also causes noise is one example.

*****

To bring back the sharpness lost with the initial noise reductions I used edge sharpening with edge detection checked. That sub tool  tames edge sharpening. The first slider, gradiant sensitivity, determines the amount of taming,  The second noise slider checks for nearby noise.  With low ISO images it has no effect; with high ISO images it increases the sharpening and can create artifacts. The last slider depends on the image and the other settings and can increase or decrease sharpness.

I followed my own advice and used the default values in the subtool. I set the sharpness, changed first level to unchanged and dragged out another contrast curve before I pulled up a 300 % view for a good look at the increased noise.

If it had been much worst I might have increased the Lum settings. Instead I went to wavelet denoise

*****

A combination of denoise and impulse noise reduction from the second detail tab--combining the two is a class 3 tip --cleaned up real noise exposed by sharpening and a few L shaped artifacts visible in 300% view.  Since the goal of this workflow was to fix the darkest area in a ISO12800 image we have to live with some loss of sharpening.

I left the bottom denoise slider at zero. Ever since I began working with wavelet I wondered why anyone would add a slider that increased the noise to a noise reduction algorithm. Jacques latest RawPedia article explained all.  Take a 30sec dark night image and the slider brings out the stars in all their glory. Another example of what seems useless or worse with one type of images my be very useful with another quite different type.

The developers didn't include all the sliders in a big joke to confuse us users. They all have a purpose and it's our job to work out the neat little photo tricks they allow.

In his RawPedia article Jacques also cautions about not expecting decent results if you use a saved pp3 on a different image. On this I think he is being over cautious. If you use a saved pp3 that was perfect for a close up of a rose in bright sunlight on one of the ninety concert hall stage shots of your favorite pop-rock band you won't be happy. But if you develop one of the stage shots and use it in the file browser to develop the other 89 stages shots you are well on the way towards fine tuning the three perfect shots you will post and brag about on your Facebook page.

*****

I used a parameter curve in the new Fine Tune tool to even out the lighting in the corridor, Now the histogram is bunched up in the center and needs to be expanded for a better black to white contrast range. So I went to CIECAM.

*****

CIECAM contrast and a touch of colorfullness fixed the problem, So how did we do?

*****

Photographybogs.com's untouched camera jpg vs RT's greatly improved jpg.

*****

A roughly neutral grey S/N improvement from about 10 to almost a 100...

*****

at the cost of a very slight loss of contrast--see the lines on the meter--that no one would notice if the two images weren't shown side by side.

 *****

A scaled down version (for the blog) of the original photographyblog jpg and the RT jpg. If you aren't impressed you must be working for an enemy big bucks image editor company. (grin)

Wavelets and Lorde

After a major professional photoshoot--the cover and feature article in Australian Vogue--Lorde, the New Zealand songwriter and singer of Royals and Pure Heroine fame--posted a selfie on Instagram. So far 148 thousand of her 2.5 million followers have loved it

My 12 year old granddaughter didn't love it. Too, too noisy. So fix it. grandpa.  (Got her RT trained since she was three and I gave her a big girl camera that took pictures of her feet and her cat and herself in a mirror)

Grandpa's two minute fix. Far less noise coupled with slightly more sharpening. The S/N was measured using ImageJ

****

The idea is to remove as much noise as possible and not worry about the sharpness. So there is nothing magic about the  91.59 luminescence. Could have been 100% if this wasn't a two minute fix and I had been paying more attention to the GUI fit on the screen.

****

Increasing the edge sharpness strength to 34 restored the original sharpness. Some small artifacts showed up as bright spots at 300% magnification. Using the denoise section of the tool removed most of them. I should note that the  tool is still being debugged and improved so these minor artifacts may not be a problem in the next build.

Final note. The denoise slider numbers in the history panel are the correct numbers. The slider settings  on the right  are the defaults when I set the sharpening strength.

To download the latest version of RawTherapee go to http://rawtherapee.com/downloads
 It's is open source, works great and will be forever free

Wavelets A Third Look

I suspect that by the time you read this blog post it will be out of date. Things are moving fast in RT wavelet land and wavelet tone mapping is galloping up behind me.

In my last post I talked about a three tool workflow using only wavelets, noise reduction and CIECAM. In this post I'm stripping down farther and using only wavelets.  Along the way I'll add my opinion about the multitude of adjustments that may panic a novice user into slider shock.

First question. "Are wavelets worth the learning curve?"  Yup and double yup.  With only a few adjustments you can do this.

****
If you aren't already aware how blogspot works click on the following images to bring up a menu at the bottom. Original size enlarges the screen shot into something more readable.

In the wavelet setting section the strength slider combines the wavelet and the original image.  I haven't found much use for the combined images but moving the slider back and forth between  0 and 100 is a quick way to see how wavelets have improved (or degraded) your image.

You can create a stack of up to 9 wavelets in any image larger than 1024 pixels. Unfortunately this includes 100% detail windows. They need to be large when I'm working on a D7000 image.  Since my 24 inch 1920 by 1080 pixel monitor doesn't have that much real estate I zoom to 100% and forget detail windows.  But if I spring for a dual monitor card for my birthday--I already have a 1024 oixel 19 inch monitor--I might rethink that strategy.

Change file size to tiled if you are using a 32 bit machine. For a 64 bit machine with a decent amount of memory full image eliminates any tile artifacts.

The D in edge quality stands for the Daubechies function. It's a step up from the Mexican Hat function used by Gimp since you can change the number of coefficients.  It that mathematical  talk leaves you confused the thing to remember is that you don't notice changes until you blow up your image to about 400%.  So if you aren't planning to use a very small crop or print a very large poster leave it at the default  setting. (See Update below. RT keeps improving.)

****

After playing around with wavelet I decided my Nikon D7000 and its kit len are a bit soft. Seven or eight clicks on the contrast + button improved things. I've also changed the default 'apply to' box from adding contrast to only the shadows and highlights to adding it to the full image. Since this improved all the D7000 images I have tested so far I'll include this as a default when I create my custom three tool profile. So use wavelets to check out your camera and lens combination to see how they perform.

If you are doing conventional development don't bother opening the preview image section. But if you occasionally slip into the Crazy Artist mode it's a double WOW place to start. I'll give an example at the bottom of this post.

****

There are three chromatic sub tools.  The pastel-saturated tool links the chroma changes to the contrast settings I've already set.  The slider shifts the effect from pastel tones to saturated tones. It works but has a limited range. In my first wavelet post I discussed both this and the nine point 'curve' that was replaced by the sliders in the next sub tool.

With  this sub tool  you can set the chroma change for each wavelet. I suspect if  I was working on another image with a range of details that  I wanted to make more colorful and another range of details that I wanted to mute this would be a useful tool. But don't expect big changes.

For landscape images I use the third sub tool.

The single slider goes from no changes to oversaturated. This will be the default in my three tool workflow.

****

Edge sharpening is much improved.  In my first post I dismissed  it with "a little goes a long way." It wasn't finished and created an artifact lover's heaven, This image shows the changes I've made so far without wavelet sharpening.

****

With 30% sharpening   All edges are sharper. But we are beginning to add sharpening noise to the walkway in front of the building. Wavelets haven't magically ended the battle between 'really-sharp' and 'lot-of-noise'.

Here is the noise at 65%, Sharp but noisy. At 100% it is much worse.

*****

My compromise in the battle was a sharpening of 43%,. Note how the blocks of the wall now jump out of the image and the sidewalk is reasonably noise free. To clean up that noise I used the denoise/ refine tool.  Unfortunately I was interrupted and forgot to save a screenshot.  The tool is simple, just a check box and three of the new combined sliders that control different denoising algorithms. The top setting is strength. The bottom is detail.

With this image level 2 worked best over all. But to remove all the noise residual in the blue sky I needed a small amount of level 1.

Leave the edge detection box unchecked.  The way it is setup now it blurs the image. This maybe because the defaults are wrong. Or it may need more work. It will, I suspect, be used to fine tune the noise and edge detection

{Update. With RT 4.2.173 this sub tool works far better--or I'm better at using it on an image that has a lot of detail. The default strength should be 0,  not 82, as I suspected.  The noise threshold slider cuts noise when used with the noise/refine tools.  The threshold high slider sharpens edges, mainly by increasing the local contrast. And it now has a visible effect on 100% images. especially when edge quality is set at D14 high.}

To finish I straightened the image to eliminate the jags on the verticals of the doors and windows

****

As a comparison I moved to the top and brought the strength slider down to 0.  Like I said earlier wavelet sharpening is very much improved.

****
I'm skipping over the gamut tool. It's supposed to have a much bigger effect on wavelet tone mapping so I save it for another post. And to be honest I haven't worked out how or why I would want to use it.

****

The residual image is, to quote, "what is left over once you extract the wavelets."  In practice that means nothing you do to the wavelets will change the residual and vice verse.  To modify the residual the wavelet tab come with a set of non wavelet tools.

As expected the first four sliders lighten or darken the shadows and highlights with the threshold sliders determining what tones are considered shadow and highlights. The contrast and chromaticity sliders also work as expected and the HH curves, set for blue sky, turn it greenish or purplish when I drag the blue control point up and down

When wavelets are finished I may discover a big reason for using this tool. But if I hadn't decided to everything with wavelet I 'd be using CIECAM for these adjustments.  The extra sliders and curves gives me much more control over the look of the final image.

****

I'm not going to attempt to illustrate  what you will see when you play around with preview setting. At the default--all wavelets viewed in all directions-- you see your image.  With the other setting you can see each individual wavelet or the stack of wavelets above and below that wavelet. Then you can scan the stack vertically, horizontally, diagonally or in all directions. Then you can set three different background: the unmodified residual image, the grayed out version and the blackened version.  Add the fact that what you see depends on such things as the local contrast of your original image and you have some idea of the number of combinations.

This is a stack, level 2 and I,  viewed vertically against a grayed residual image.

****

And this is the same thing viewed vertically.  A bit of a difference.

****
Some time after midnight as I was about close up, my Crazy Artist muse conked me on the head and yelled in my ear. "Forget this one tool workflow crap. artist boy. Open up all them curves and sliders and buttons. Move them control points around. Go for the ugly. Go for the weird. Go for the crazy crazy, crazy til you got something so crazy I'll let you to put my name on it. And if you don't get it exactly right first time, don't worry. I'll be back to make you do it again. Cause I'm an addictive muse, artist boy."

Best viewed original size

While this post was written using RT 4.2.151 get RT 4.2.173 from the download page
There's been some fixes and improvement but the layout is unchanged.

http://rawtherapee.com/downloads

From Photo to Line Drawing....Without Leaving RawTherapee .

The jpg of the girl in the fur outfit has a moderate resolution: 804 by 629 pixels. While working with high resolution Raw files will generate interesting images, if you are after a line drawing medium resolution jpgs works best. That way you can view your image at 100 percent. You will be creating what would be called artifacts in a normal workflow and it is easy to go a bit too far with the sliders.

Step one

Start by upping the exposure compensation. For a realistic line drawing keep detail in at least one of the color channel in areas such as the girl's face. In other areas that will be paper white in the line drawing such as the girl's skirt you can push all channel to 100 percent,

Step two

Tone map the jpg to create edges without turning shadows into blotchy artifacts in area such as the girl's face.

Step three

If you like your original colors  this and the next step are optional. Using the three color temperature sliders produces a wide range of color shifts ranging from a total color cast to a blue tinting of the edges created in step two.

Step four

Again this step is optional.  The channel mixer switches the colors towards green.

Now another big reason for using roughly 800 pixel resolution jpgs, The bottom sliders in the contrast-by-detail tool creates hard edges. With higher resolution images the effect is nowhere as strong.

My original final image.

A water color perhaps rather than a line drawing but still an attractive image. Because of the interactions between the tools pushing beyond this didn't work out. But after I slept on the problem when I woke up I realize that if I did a second pass stating with my painterly jpg I wouldn't be fighting any interactions.

Repeating step one and two--more exposure, more tone mapping, and then setting the saturation down to -100 created this  line drawing.

Edit

Variations on a theme.

Toss in the black and white tool channel mixer and you have a ton of manipulations to play with. Here are two examples

These images were created with RawTherapee build 4.2.77. It is open source and forever free and can be downloaded at  http://rawtherapee.com/downloads

Mini Workflow Spot Color

The original image of a couple kids being photographed by their mother.

All colors except for blue are desaturated using the saturation curve of the HSV tool.

Fine tune in CIECAM using the JS Lightness and Saturation algorithm.

That's it folks.

But, of course, with RT editing nothing is ever finished. So here is the new stuff you can create when you switch to the all algorithm and start draggin' them sliders around.  The hidden artist titles this 'Cindy's Nightmare - Do you have to tell wicked step moma you caught me sneaking out to the Prince's Rave?'

Main points
The hue slider gives a full range of of colors
When  ISO6400 noise is all the same color it becomes artistic texture. Fine tune that with the Noise Reduction Chromatic sliders
The other CIECAM sliders work pretty much as expected but with different intensities. For example the Q brightness slider is stronger than the L lightness slider and at 100% washes everything except the colors to white

Happy exploring your own Hidden Artist

Processed with RawTherapee 4.0.12.165  which can be found at http://rawtherapee.com/downloads

First Look at RawTherapee's new B&W conversion tools

Been a while since my last tutorial. It's partly because of the press of other commitments, also known as procrastination.  And partly because of a real medical condition effecting my eyesight brought on by, among other things, being too friendly with a high powered UV laser during my misspent youth. But that problem seems to have stabilize so it is unlikely you will ever see an image of me sprawled on the the sidewalk next to a stack of   blurry prints, an old can and a sign saying 'please support this near blind photographer.'

"Blind photographer" turns out to be not as wild a joke as I originally thought. 
 A google search brought up
 http://lens.blogs.nytimes.com/2012/09/18/visions-of-a-blind-photographer/

With luck this excellent essay should come up on your screen. Unless the NY Times has changed its policy everybody has 20 free articles a month before they must subscribe.

The big news in RawTherapee is a Black and White tool kit that 'blows the pants off' every other image editor out there. Or should that idiom be 'blows the pixels off the print'? With algorithms by Jacques, the same developer who gave us CIECAM02, it can both mimic a film darkroom while still providing all the quirky effects my hidden artist will learn to love.

The first image I will convert is courtesy of Charlotte-- the artistic kid who has been featured in earlier blog posts. It's her entry in her 1st grade art exhibit six years ago. Would have been the grand prize winner if the first grade was more competitive and awarded grand prizes.

The black and white tools are found in the color tab.

The first of three, Desaturation, produced this. As with a greyscale in Gimp or Photoshop all RGB values are identical. While sliders can adjust the gamma of each channel, the tool's two curves provide most of the adjustments. The 'before' curve works on the RGB colors before the conversion to BW. The 'after' curve works on the 'L' channel after the conversionl So it changes the image differently. Like all other RT curves there are four choices--linear, custom, parametric and control cage.

This mild parametric curve did darken the shadows and highlights slightly  even if its effects aren't blindingly obvious/

This far more drastic custom 'after' curve darkened the image more than I'd liked so I tried to fix it by upped the exposure.

This switch to an almost negative image is not what I expected to see. It's not a bug. There is nothing boringly linear about Jacques's algorithms. Tool tips warn about artifacts when you push sliders and curves too hard. My hidden artist thinks differently and proclaims.  'Not artifacts! Great new features!!' And if linear is your thing, just don't push the sliders and curves so hard, Or move on to the second tool- the Luminance Equalizer

Unlike the Desaturation tool that works on the whole image the Luminace Equalizer lightens or darkens specific colors.  Such as the golden foliage in this jpg.

The original B andW image is dark and a bit blah so--

I grabbed the yellow color and lighten it by pulling it up in the flat field editor. The latest version of this editor has a great enhancement--a bar at the top that shows what color you are working on. Drag the cursor to the left and the color shifts towards red. Drag it toward the right and it shifts towards green.  And if you drag it down you darken rather than lighten the color.

Like all of the B and W tools, you can also tweak your image with a curve.

This 'after' curve darkened the the fallen leaves on the ground.

Both of these tools are useful but the real action is in the Channel Mixer. Lets see what I can do with this shot of trees in a wild life preserve on the edge of Lower Mud Lake.

I have software color filters to work with. They mimic the physical filters you would screw on the front of a lens when using film. As an example the #33 dark red filter I used to carry in my camera bag would block the blue and darken a sky.

I also have presets. The four labeled Channel Mixer ... down near the bottom are fully adjustable. The others have only gamma sliders and are adjusted by curves, the color filers or RGB sliders outside the the B and W tool section.

I'm interested in a look that mimic 19th century wet plate photos with their totally blown skies.
The orthochromatic preset seemed to be a good place to start. It cut the red to 0% and lighten the sky but there was still too much green.

So I added a blue filter.  While it cut the green to 3.5% and reduced the trees and field to a silhouette the sky still had detail.

An 'after' curve lighten the image. Better but it still wasn't quite the effect I was after.

In this build the labeling of Channel Mixer Absolute ROYGCBPM is confusing. The ordering of the eight color sliders are red, green, blue, then orange, yellow, cyan, purple and magenta.

Bringing the yellow slider down to -100 lightens the sky and is nearly the effect I'm looking for.

Moving the orange slide up to 102 darkens the sky.

But pushing the orange slider to 129 now lightens both the sky and image.

Move it one more step to 130 and the lights go out. They stay out until about 185

Then the image flips over to white. It started to come back until at 200 I ran out of orange slider.

Then by pushing the magenta slider to 170 I hit the 19th century wet plate look I was after--totally blown skies coupled with excellent tonality in the trees and fields.

Is all this a major bug? Apparently not. If you look at the now closed issue 2010 at https://code.google.com/p/rawtherapee/issues/detail?id=2010  you can read two months of developers' posts on non linearities and other matters. The consensus appears to be 'this is the way the algorithms work so learn to live with them'.

You can also uncheck the 'adjust complimentary colors' box and the sliders will work very differently. But that will be part of another post. It's time to start thinking about digging out from last nights snow storm.

Both at 32bit windows and  a mac version of build 203 are now up in the RT download page at
http://rawtherapee.com/downloads