Nikon D800: The best sensor analyzed on DxOMark (so far?)

[Grant Atkinson]

I have owned and shot extensively with the Canon350D, 30D, 40D,50D, 7D, 1Dmk3, 1Dmk4 and 5Dmk2. I have also shot quite a bit with the Nikon D3, D3s, the D300 and the D7000 in the field.
I spent long hours on DXO Mark trying to learn how to correlate what I was seeing in terms of image quality, noise, etc on the images I was working on from different cameras, with what the DXO measurement graphs showed in terms of numbers. For a long time I felt that by looking at the actual Measurement graphs for a particular camera, you could get a good idea of what its real image quality would be like, and how it would compare to another that I might already know well. I had total confidence in the DXO Mark graphs.
For the last couple of years however, I find myself a bit perplexed by my own perceptions of how cameras I either own or shoot a lot with, correlate to what DXO Mark show on their website.
I will cite an example here: If you take the Canon 1Dmk4 and the Nikon D7000, and look at the overall DXO Mark score, you will see that the D7000 at 80 outscores the 1Dmk4 at only 74. However, if you select those two cameras in the Compare Sensors tab, and then inspect the Measurement graphs themselves, there are five different graphs to look at. They are ISO Sensitivity, Signal to Noise Ratio, Tonal Range, Dynamic Range and Colour Sensitivity. For all 4 of the first measurement categories, the graphs show the 1Dmk4 as being better or equal to the D7000, and in some instances significantly better, almost all across the ISO range. The D7000 shows superior Colour Sensitivity and better DR at very low iso settings.
I find it quite difficult to understand how DXO Mark can provide such measurements where the camera with larger pixels and high-end electronics (the Mk4) is scoring higher in all but one test, which is to be expected and yet the is rated inferior in the overall score? Clearly there are many other parameters that they are taking into account before giving their final rating that we do not know about.
My own experience in using both of these cameras is that the 1Dmk4 is a little better (and it should be given its level of specification) when it comes to noise, and image quality, than the D7000.
Instances like this make it hard for me to take their overall rankings seriously, even though three years back I was a believer...
cheers
Grant

[Forgiss - Sean Nel]

In complete agreement. I don't see the correlations in real world terms. I can't argue with their test results, I am most probably just not seeing the same important factors as they are.

If I look at their results comparing a current 5DmkIII, a 5DmkII and a D3s, they only differ three points in total score. Their graph shows the D3s better at every point of both those cameras, but their scores show, for instance, the 5DmkII and mkIII better at colour depth.

I just don't get it anymore. I think the tool struggles to accurately compare modern sensors

[Francois Retief]

I should probably be out there taking photos instead of looking at graphs of cameras on the DXO website ! However, it is quite interesting and I did not manage to go out shooting this morning. I have a few comments and thoughts on the DXO scores.

* The DXO score is a test of the camera sensor (and not the camera) performed under certain conditions and taking certain readings. It is one objective method to compare sensor performance - not the only method.
* DXO seems like a professional organization so I don't doubt that their measurements are accurate. It is how you interpret the measurements that are important.
* To get the full picture you need to compare the graphs and not the overall scores. To make things simple they have to "dumb it down" to a single score - that is what makes the headlines.
* The D800 gets its high score mainly from Dynamic range where it outperforms the 5D3 by 14.4 EV to 11.7 EV. This represent a point on the graph taken at an ISO of 100. If we move on the graph to ISO's over 1000 the 5D3 is on par with the D800 and it gets better than the D800 at higher ISO's
* For ISO's up to 3200 the 5D3 is only about a 1/2 stop better than the 5D2 as far as low light performance goes. After that it improves to 1 stop and more.
* Looking at the graphs you can clearly see the improvement of the 5D3 over the 5D2 - this is not reflected in the overall sensor score difference of 81 and 79.

A camera is more than the sensor - just as a car is more than just the engine. Just as you won't (I hope ) buy a car looking only at the engine specs, you should not buy a camera based only on a sensor score. It is a consideration but should not be the only measure. You also need to look at autofocus performance, frame rate, metering, ease of use, functionality, etc, etc. In addition to that you needs would differ depending on what you shoot i.e. only landscapes and studio or only birds/wildlife or sports or a combination of all. Then you would consider the brand - what lenses are available, service and support, availability of information about the camera,etc. etc.

Then the most important of all - if you have a camera go out there and use it. A Blackberry or Iphone is better than a 5D3, D800 or D4 if you don't use them !

[elsahoffmann]

[QUOTE=Francois Retief;259741]
Then the most important of all - if you have a camera go out there and use it. A Blackberry or Iphone is better than a 5D3, D800 or D4 if you don't use them ![/QUOTE

exactly!!!!!!!!!!!

[BrerFox]

* To get the full picture you need to compare the graphs and not the overall scores.

This is the real crunch, isn't it. And these graphs have given me a far better idea of what to expect from a 5D3, in comparison
to the bodies I own already than any subjective test I have seen. Combining the graphs into a single, weighted figure is obviously
flawed, but the graphs work very well for me.
Elsa, yes, the purpose of the camera is to be used, but the specs. tell you where it can be used. I, for one, could never use the
5D2 simply because its autofocus specs are not up to my needs. If I had bought one, I simply could not have used it - I would
choose my 7D over the 5D2 every time. I regard a camera as being an extension of my body - another limb, almost. And in the
same way that I get frustrated wearing clothes that do not fit, I get equally frustrated with cameras that don't. Specs help me a lot
in narrowing down the cameras I should be considering.

[Forgiss - Sean Nel]

Oh, I very much agree Francois! DxO is a sensor test, nothing more, and if it tested a 40mp sensor with no noise and 20 stops dynamic range in -2EV they would rate it exceptionally high even though it takes 40 secs to expose the shot.

I know the overall points score is not close to a match for anything, it's the nature of points:

Low light monster scores: 6, 7, 5, 3, 4 = 25 (nothing beats it in low light sports)
Landscape Dream scores: 3, 4, 4, 8, 7 = 24 (nothing beats it in dynamic range)
Entry level camera scores: 5, 5, 5, 5, 5, = 25 (beats nothing in anything)

According to the "points" an entry level camera could happily beat the best pro bodies that is geared for niche markets, so yes. I agree with you completely.

Why I say I don't think I understand it is the discrepancies with real world use. For Instance, most review sites claim matched Low Light/High ISO performance or slight improvement between the D4 and D3s, however DxOmark rates it as 10% worse?

[Edit: some faith restored] Just read the f/stoppers review and they also think the D4 is noisier than the D3s [end edit]

Unfortunately, as much as I want to go out and just shoot, I have to study the charts and image examples, and all that nonsense because not a day goes by that I am asked about which camera is best (with my answer being "best for...what?"), and then I have to be able to give an honest, true and informed opinion. DXOmark was a great resource for a few information points, but now it feels like the system broke.

I am not knocking DxO, I am lamenting the fact that it appears I can't use the resource effectively anymore...

[Francois Retief]

Sean - I understand that it is part of your job to "pixel peep" and study all these charts and image examples !!!!

I must say I also find some of the DXO results very strange - the D4 has a higher overall score than the D3s (89 vs 82) BUT on all the graphs the performance is about the same except for Dynamic range where the D4 outperforms the D3 but only if the ISO is lower than about 2000. I understand what you mean - if all other functions on the two cameras are almost equal (hypothetically speaking of course ) and you "trust" DXO then how do you convince anyone to upgrade to a D4 ? By telling them the only improvement is dynamic range below 2000 ISO and then the 1 EV difference is so small anyway that you won't notice it 99% of the time !

I have a great camera to take photos with, the DXO investigation was an interesting diversion but now I promised myself that I will focus on taking photos out there in the real world and not spend my time on the DXO website ...

[Forgiss - Sean Nel]

...I have a great camera to take photos with, the DXO investigation was an interesting diversion but now I promised myself that I will focus on taking photos out there in the real world and not spend my time on the DXO website ...

Hahahaha... after writing my last post, I decided the same... but my battery is flat!? so now I am charging and writing more posts

[Henk Coetzee]

One thing which worries me about DxO scores is how they calculate a total. This individual measurements are understandable. The totals don't always make that much sense.

FWIW, today's post on The Online Photographer makes some very valid points.

[BrerFox]

Thanks, Henk - that is a good artice and thought-provoking.
Some of the comments are also thought-provoking.
This is one that I think sums it up very well:
" Better tools give us more options, more possibilities. They don't guarantee better results, but they do make new things possible to do and they give more choices for the photographer.On a side note, it's also good that there's serious testing such as that of DxOMark available. I'd loathe to have photography to be dominated by the sort of mysticism that is prevalent in amateur audio."