Noisewise, Is My Camera Meeting Spec?

Noise spec? you ask.  Point to where that is printed on the camera's spec sheet, you demand.  Sorry, camera manufacturers don't  spec their noise or more accurately their signal to noise ratios. Why? Maybe it's because some of us meany pixel peepers might take the number seriously and start demanding refunds. Or, to be more charitable, it might be because accurately measuring a camera's S/N ratio takes more than a look at a computer screen?

Physics-wise, S/N ratios are complicated. Photon shot noise, addition in quadrature, Poisson  statistics--those sort of things. Complications on top of complications.

A more reasonable question would be "Is my camera as good as the camera that was used  to take the photos for the review I read on my favorite photo site? The one that convinced me to buy the camera?"

If you are lucky, yes. More likely, no. Any marketing manager worth his corner office will make sure the cameras sent to review sites were hand picked for performance. But your camera should be close or you do have the right to demand your pixel peepin' refund.

This came up in the forums recently. A new user had bought a 'bridge' camera or, as I used to call them before marketing folks invented the name, a super zoom.  Those types of cameras have little sensors since, among other things, their light weight and inexpensive super zoom lenses only make little circles of focused light. Since they also pack about 12 megapixels into their little sensor--lets say the trade offs are not favoring S/N ratios and low light performance.

Our new user picked the right place to fix his noise problems even though his expectations were originally too high. RawTherapee has great noise reduction tools but there are limitations. So a question came up. Was his camera's S/N ratio within 'spec'?

Here is the workflow I used to work out the answer by using a comparison image, RawTherapee and ImageJ

I've already posted a tutorial on how to create a noise profile in ImageJ so I won't repeat the steps here.  http://scribble-jpc.blogspot.com/2013/03/crreating-noise-profiles.html

Imaging Resource is the place to go for your comparison images. http://www.imaging-resource.com/ They have created a massive data base of reference images going back to the days when a 2 megapixel camera equaled a $1000 investment. And they did things right from the start, controlling details, such as consistent lighting, needed to create reference images that highlight real differences between individual camera brands.

To download your comparison image pick your camera in the review pages, go to the sample tab and then the sample image page.

Near bottom and after a multitude of jpg images you will find Raw downloads of their multi-image test shot taken with your camera's various ISO settings. It has the Macbeth color chart we will use. I downloaded the ISO6400 version  to compare it with  the bowling alley picture I blogged about earlier.
 http://scribble-jpc.blogspot.com/2013/03/shoot-at-iso21000-print-at-30-by-45.html

Using the neutral profile I converted both NEFs  into jpgs before loading them into ImageJ. That conversion insures I was doing a real apples to apples experiment. If your camera doesn't take RAW files, obviously you must use jpgs but then your test image must have been taken with the same setting as the reference image for any meaningful results.

Pick the closest match you can find on your image, the wall behind Rhianna, and on the Imaging Resouces' multi image, the #4 gray scale box.  Run your profiles. Compare the graphs.

That's it. Twenty minutes time max and you know how well your camera is working. Take a few more measurement, let ImageJ do the signal averaging and you have an accurate S/N number.  Then you can brag about your new camera's performance in the forums. Or. more important, toss down a hard copy on the service desk if your camera needs fixing.

As for the new user who just posted his "Thanks" -- "You're more than welcome since your questions inspired this tutorial."

You can find RawTherapee 4.0.11.1 here. The package includes an updated manual

http://rawtherapee.com/blog/rawtherapee-4.0.11-released

A Study in White and Blue

On a bright and warm May day earlier this week-today is grey and chilly- I went to the Olbrich Gardens to field test new optics. Last Saturday I had bought an oldish Nikkor 50mm f1.8 lens that uses my D7000's internal motor to focus along with a set of extension tubes with all the electrical and mechanical connections needed  to completely control the lens. The hidden artist do-something-different challenge I took on was to leave my zoom lenses in the camera bag and shoot with only the 50 mm lens.

The magnolia trees were in bloom and the sky was blue with wispy clouds. My original shot with RT's neutral profile. Obviously it could use a dose of RT magic.

After a white balance on the magnolia leaves and a default profile.  This version is better but the hidden artist within wanted a bit more ump

The HSV Equalizer tool has been updated  and is now less finicky to use. Dragging its blue bar up or down to change the value or lightness of the blue in the sky seemed appropriate. Here is up. It didn't make a massive amount of difference since the sky's value was already high.

 And here is down which did. Notice how the  blue channel has shifted to the left in the histogram

With the value curve reset I played with the blue saturation curve. Here is up.

And here is down. 

With this image it may not be immediately obvious but a judicious use of both the value and saturation curves is a great way to improve skies without going over the top like I'm doing now. The Hue curve rotates the color wheel to create totally aliens skies. This green version is one of many.

After settling on a moderate increase in the sky's value I went on to the CIECAM02 tool. A large contrast boost of 80 made both the sky and the tree far more dramatic.

And to lighten the clouds  I upped the brightness to 25.

To bring down the blues I played around with the Colorfullness curve.

And as a final comparison here is where I stated once again.

RT's color correction tools are the best around, period. So load up an image the could use a little sky magic and start dragging curves and sliders. I predict you will like what you discover.

Hidden Artist Strikes Again

I started with this-a 1949 cover for the Saturday Evening Post painted by Norman Rockwell

After I did my camera and photographer twisting and twirling I loaded the image into RT using the POP  profile.  This new addition to RT uses tonemapping and CIECAM02 to add, surprise, surprise, pop to your image.

To prevent one problem, jet black splotches that appear in highly over exposed areas when tonemapping and CIECAM02 are used together, Jacques, the developer who added CIECAN02 to our toolkit (applause, applause) made changes recently.  He compressed the highlights for well exposed images taken with bright scene illumination, the highlighted value of 2000 standard candles per meter squared.

 While that fixed one problem it added a second. For less well laminated scenes like indoor scenes with over exposed vistas viewed through windows, or like the low illumination I used to create this image, the histograms never go above 75%.  White areas end up too grey.

 The fix is blindingly obvious if you understand the intricate working of CIECAM02. You adjust the scene illumination slider to match the image. The histogram moves to the right; you stop when it starts to clip.  But the rest of us who think a standard candle is the long thin one you push into a brass candle stick we tend to miss what's needed to be done.

Yesterday afternoon I discovered that undocumented fix. Not by some brilliance on my part. In a not-so-secret forum where RT developers hang out and talk RT programming talk, this was being discussed in issue 1827. So last night, me, a lowly user, dared to tell Jacques, the developer, that his technically correct tool tip needed to be rewritten. In the middle of our night and his morning-the time difference between Wisconsin and France-Jacques wrote back saying of course he would do it

Took less than 8 hours. So if any users of commercial programs such as Lightroom happen to stumble over this blog please put in a request for the perfect fix you've been waiting to use. Then let us know how many weeks/months/years/eons it takes to get an answer back. RAWTherapee might bring on slider shock and have its quirks, but its developers listen. And when it rocks, it Rocks!

EDIT.  An Oops. Or semi Oops. What was less than blindingly obvious when I wrote this was that the real  cause of the problem was a bug in the CIECAM02  calculations. With build 4.0.10.69 that has been fixed, the problem no longer exist and the histograms no longer hang up at 75 percent no matter where you set the illumination slider.

A two slider adjustment in scene illumination and exposure moved the red channel to the clipping point and lightened the image.

A CIECAM02 contrast boost darkened the edges and clipped the red channel

.
A wild LAB custom CC curve made big changes to the image and brought all the channels together. Not the usual way to set a white balance.

A final scene illumination of 145 cd/m2 sets the overall image whiteness.

Two curve and slider adjustments didn't change the image that much. Running the chromaticity to the max, however, gave me a wild effect I could use.

The final masterpiece ready to be printed and matted for the masterclass.

And because it is also neat the black and white version created by clicking  BW tone mapping in the LAB section.

The Artist Hidden Within

I'm taking a master class on how to drag the 'artist hidden within' out into the daylight. So for the next several weeks that hidden artist will be raising his sarky head and inflicting his 'masterpieces' upon the innocent blogosphere.

As any regular reader of this blog knows, the non hidden artist's aesthetic tends towards photographic perfection, the sort you pixel peek at 100% or higher to see. Accurate colors, perfect sharpness, noise freedom has reigned here.

The hidden artist, on the other hand, favors this.



The original Playboy style image that posed for the 'artist within' is from a 40 year old Petersen's Master of Contemporary Photography book about Bert Stern.  If the name Bert Stern isn't instantaneously recognizable  he's the photographer whose 1950's ads turned vodka, then a cold war Russian commie pinko exotic, into an all American drink. This really dry vodka martini ad was worth $4000 plus travel expenses in 1955 bucks. If his reputation has slipped a little since then he lived very well in his time.

Enough dry history. Here is the 'hidden artist's' RT workflow.

First select a colorful page or double page image to mangle. Set the camera to manual, to ISO100 or lower, to between 2 to 10 sec exposure and to whatever iris opening gives a reasonable photo. Move camera while snapping the photo. Better yet, twist, roll and yaw the photographer while snapping  the photo. Since you will have no idea what you have on the card, do this a dozen or more times. Load  the most interesting photo into RAWTherapee. Or in this case, one that didn't violate Blogspot's prohibition of full frontal nudity. Finally start the masterpiece making.

This brought out the color in the wide tie, the subject of the photo.

A hue slider adjustment that shifted the colors towards green. The CIECAM02 algorithm is 'all'.

Playing around with the CIECAM02 sliders brought us to this color mixture.

Since the colors aren't changing much 'the artist within'  turned this into a tiff file. Then he loaded that back into RT for further processing.

This lighten the background.

With this 'the artist within' laid down his mouse.  And left it to 'the outer guy' to write this blog post. He liked this version the best.

RAWTherapee 4.0.10.36 is now here. Besides speedups and bug fixes, it offers a new set of packaged profiles and two new demosaicing algorithms intended  for use with HighISO images. (Big Grin)

http://www.visualbakery.com/RawTherapee/Downloads.aspx

Kindergarten Snow Day---Another Quickie Workflow

The history of this less than impressive image started with flickr mail about non CPU manual lenses. I had accumulated a collection from garage sales, Craig's lists and on-line non ebay auction sites back when I was shooting with a Nikon D60. One lens was an Osawa 300mm f5.5. 

If I remember right curiosity drove that purchase.  Osawa?  Never heard the name. What kind of lens was that?  Osawa turned out to be a short lived 35mm camera maker from the 1970's. But a decent lens maker who, last time I googled, was still making lens for medium format cameras.

The lens did not disappoint and became part of my D60 kit. I posted some shots on flickr but when I bought the D7000 the lens was packed away and eventually buried under other boxes of good junk being sorted. Wife has ordered what threatens to turn into a massive Spring cleaning.  So when I received the email from another Nikon lens collector asking how it worked on a D7000, I had to find it. Which delayed any lens testing to the day of the latest Midwest mini-blizzard. 

The test target was house a hundred yards away from an upstairs bedroom window. The test was: yes the Osawa mounted and yes it took pictures. The fact that kids enjoying a snow day from kindergarten  wandered into the image part of frame and a hunk of out of focus curtain half filled the rest was coincidental. That the image ended up on my computer along with a directory full of more important shots was equally coincidental. That it wasn't immediately stripped down to loose pixels and tossed away into the reject bucket wasn't coincidental.

Recently I've been reading about photographic history and aesthetics.  The super sharp, noise free, tonally perfect aesthetic embraced by most of us RT users--why else would we put up with this many check boxes and sliders and even demand more -- can be traced back to photographers like Ansel Adams and Edward Weston. Both spent inordinate amount of time and effort in the darkroom producing perfect prints. But there were other aesthetics: the pictorialism of Stieglitz's Camera Works, the abstractions of Minor White's early Aperture, the photo montages of Rodchenko revolutionary posters--and so on.

 Let's call this an example of  Scribble's ...Colorism.

An auto levels exposure correction. If I had wanted to go for an Ansel Adams Zone System tonality I would use the black point slider to expand the histogram to fill the empty left section.

As good a time as any to crop. Since a square frame worked I fixed the ratio and invoked the rule of thirds. The big difference in the histogram came from cropping out the curtain.

A white balance in a snow scene is always a good idea even though in this image it didn't make much of a difference.

Now the Colorism created with CIECAM02.  The Contrast (up) and Brightness (down)  moved the snow into the unclipped but close to pure white area of the histogram.  The Colorfulness slider fully colorized our models, human and canine, and gave the image its snap.

Now a mini confession. My first workflow wasn't as quick and clean as this tutorials claims. I played around with slider and curve combinations, made a bunch of jpgs, sent one off to flickr and even collected a comment among the views. Wasn't til the end of the mini-blizzard and the snow blowing was over that I decided this might be worth a tutorial. So I duplicated the settings from memory.  And ended up liking my first version better.

A hint. If you ever need to duplicate an image or workflow and find yourself mentally kicking yourself  for not saving a profile, load the pp3 file of your good jpg into notepad and duplicate the settings. That will usually get you back to where you've been before.

Not my usual image but --sometimes you have to go wild and live dangerously.

Slider Shock or What Do I Do Now

You have  RT on the screen in front of you. You have read the first pages of the manual and now have the file browsers loaded with a directory full of images. You have selected your first RAW file. You are ready to fix problems.  Your snapshot will become a sent-to-all-the-grandmas minor masterpiece.

Then it hits you. Big Time.  Slider Shock!!. You think "Never Never in a Zillion years will I Learn how to use All those Sliders and Check Boxes."!! You mutter "I want my money back"! Then you remember RawTherapee is free.

Happen to everyone. Even me. And back then there were a bunch less sliders. But by now I've discovered the big RT secret.  Learning how to use all the sliders is for hang-on-the-wall-and-be-awarded-the-ribbon masterpieces. For the send-to-grandma ones you only use a few sliders. As for the rest--I must confess it wasn't until my last post that I needed to know much of anything about what was going on inside the RAW section. You can pick up most things as you go along.

This is a common exposure problem. Too light outside the window. Too dark inside the kitchen, The only thing correctly exposed  is the flowers on the window sill.  

The send-to-grandma workflow in seven easy steps:

Exposure compensation to 1.00.  This lightens Charlotte-the grandma interesting subject--but blows out the outside  foliage.

Enable HighLight Reconstruction. Because we are working with sky peaking through foliage I used the blend mode. The highlight recovery sliders-out of the picture- are at their default settings.

Now we are dealing with what I am beginning to think of as the twin power tools of RT. The four entries in  the history box appears when you set up Tone Mapping and CIECAM02 to work together. Just follow the steps in the Tone Mapping tool tip.

I used the control cage version of the Brightness curve and dragged it up until the snap was brighter than I wanted.

I fine tuned the snap with the Brightness and Colorfulness sliders. This shows the setting I decided to go with.

Rather than loading the snap into the queue. I copied my new procedure to the clipboard.

Finally I went back to the file browser, selected raw files with similar exposure problems, pasted my procedure and sent them all to the queue to be turned into grandma happy jpgs ready to be printed or emailed.

A couple of the jpgs that came out of the queue.

Not hard at all,  If you want to do additional work --a crop for instance--load up the raw files again. They will come up in 'last saved' mode ready to be worked on.

Rescuing REALLY IMPORTANT snapshots from overexposure

Last summer Charlotte took her mother, her grandmother and me on a time travel adventure to Old World Wisconsin. She'd been there before on a school outing and promised to show us the really good places. I could even take my camera if I promised to document our adventures.

One of the good places was the blacksmith's shop where, as the only kid, she got to do all the fun stuff like working the air pump to heat up the forge fire so the blacksmith could bend an iron bar into a hook. She  even got to buy S hooks like she help make to hang up her REALLY IMPORTANT stuff in her bedroom As promised I documented her every move with my D7000  set at ISO2000.

Then it was across the road to the German farmhouse where it was washing day. She got to help here too, doing everything from pounding a shirt in the rinsing tub to twisting one end as she and the old timer haus frau wrung it out. Again, as promised, I documented every step. Then, horror of horrors, I discovered I'd forgotten to reset the camera's ISO. All the pictures of this REALLY IMPORTANT part of the adventure were overexposed!!

Can they be rescued? Ask RawDigger. Like RawTherapee it's a free download found at  http://www.rawdigger.com/  It's easy to use and will tell you all the things about your raw photos that you never thought to ask.

RawTherapee's highlight reconstruction works by using the data in the less damaged RGBG channels to fix the data in the totally blown channels. In this RGB render mode, you are looking at a rough color image created without a true demosaic or white balance  You can also look at the composite mode which is very green or at the individual color channels.

The red splotches show the blown areas: in lighter red where there is data in at least one channel; in darker red where all data is blown out. RawDigger will also shows underexposed areas using blue splotches.

The program has another feature  worth the downloading time if you are ever faced with a directory of overexposed  images. With a CTRL + left or right key I quickly stepped through the overexposed NEF files to grade them from 'easy fix' to 'this will take serious work in GIMP or Photoshop.'

I graded this image as moderately difficult. While there are numerous bright red patches that will cause problems they are not in areas that would do major damage if they didn't fully clean up. So for this tutorial I'll concentrate on Charlotte's shoulder and an area in the closest tub.

Next question:  what highlight reconstruct method should I use?  The choices are:

Luminance Recovery

CIELab Blending

Blend

and Color Propagation

With this image Color Propagation is so obviously superior--the wood grain in the tub, the strap on the shoulder-- you might ask why are the other methods still on the menu. One problem is Color Propagation can produce a reddish color cast. Something that is fine for skin tones but not so fine for violet tinted skies. A second problem is it may not work as well with other camera brands.

It certain doesn't work with all camera profiles. This image would have been an 'easy fix' if it wasn't for the way Nikon compresses data to a supposedly lossless raw file. Except for a few speckles the blue channel is not blown. At the moment RT doesn't have a D60 input profile so I cleaned it up using the camera standard profile.

And this interesting monstrosity  is what happened when I switched profiles to the Nikon D60 Adobe Standard.

All profiles are not created equal.

Finally this is my rescue workflow.

Reduce the exposure. With this photo by about -2 EV.

Go to Highlight Reconstruction with Color Propagation and let RT do its magic.

Charlotte's face is too dark.  Go  to CIECAM02 and added a parametric Lightness curve

That's it, Not perfect but from what I saw with RawDigger what I expected  Once all color channels are blown the data is gone forever and not even RT can bring  back fine detail like the wood grain in the bucket.. But RT can reconstruct colors in areas like Charlotte's shoulder where there is no important detail,  Plus in the final rendering the transitions are not as sharp as they look in the view boxes.

Tell the truth. If you hadn't been following the tutorial  would you have noticed Charlotte's shoulder or the wood grain in the bucket in this final image.

I'd like to thank iliasG and DrSlony for the advice they gave me. While any mistakes here are mine, if they hadn't set me straight on a few key matters, lets just say things would have been embarrassing. Some of my assumptions were not, in political media jargon, 'true facts.'

edit- DrSlony, an RT developer, sent me his workflow;

Scribble I use the "Blend" highlight reconstruction mode by default. Most of my shots are landscapes, which means I usually have time to plan the shot and there are not unexpected over-exposures. The Blend mode makes the transitions from well-exposed to over-exposed smooth and it keeps the overexposed area white, which is usually what I want. Color Propagation is amazing at reconstructing overexposed skin, so whenever that's the case I use CP. CP is also the only method for which you can move the Highlight Recovery slider well past 100. When you do that with the other methods, the overexposed areas quickly get dirty grey or some other form of ugly, but with CP you can easily go over 200 or even 300 and get a great looking result. CP fails at large overexposed areas, and it's slower than the others.

Anyone else who wants to contribute his or hers experience with RT is welcome to join in.


You can find this build of RT at-
http://www.visualbakery.com/RawTherapee/Downloads.aspx
The online manual is at-
https://docs.google.com/document/d/1DHLb_6xNQsEInxiuU8pz1-sWNinnj09bpBUA4_Vl8w8/edit