If you regularly follow tech news, you might have come across the name DxOMark. That being said, there's only a few people out there who know and understand the origin of these rankings - these are taken very seriously by both smartphone users and manufacturers alike. Taking advantage of their presence at this year's MWC, I was able to learn more about how exactly DxOMark works.
Regularly highlighted in the press, the DxOMark scores are now the industry standard for camera and lens image quality measurements and ratings. As a result, these scores have become key criteria for many people who are in the market for a good camera phone. The people behind these tests are DxO, a French company based in Paris, which was launched in 2003. Its core business is image quality, however its activities are not only limited to DxOMark scores.
More than just scores
The company offers a wide range of software solutions for professionals and experienced photographers. For many companies, it's simply one of the best reference points in determining image quality. The company also offers its consulting services to manufacturers to help them design their products so they can improve the quality of their images. DxO thus shares its methodology and lab facilities to advise companies. Since September 2015, the French firm has also marketed its own external camera for smartphones, the DxO ONE, which is available for around $500.
Each phone score is the result of a 7 day testing period, which includes taking over 400 photos and 30 videos
DxOMark is a showcase for DxO. It enables DxO to demonstrate its knowledge and mastery of image quality so that consumers will gravitate towards its product. The company then tests the quality of the images produced by cameras, lenses and smartphones. For the first two areas, DxO performs a so-called objective measurement (from the RAW files) to get a better idea about a device's potential. For smartphones, the process is slightly different. DxO performs a combination of objective measurements in its lab and experiments (from jpeg and mpeg files). Each phone score is the result of a 7 day testing period, which includes taking over 400 photos and 30 videos.
An objective ranking?
Of course, many of their biggest clients are smartphone manufacturers. The photographic experience which is offered by new devices is indeed one of the most divisive consumer elements in the smartphone market today. MWC 2017 only served to reinforce its importance, as some of the most talked about selling points for new devices were their cameras and image quality. Huawei highlighted its partnership with Leica during the presentation of the new Huawei P10, LG focused its attention on the new dual photo sensor in its flagship G6, and Sony introduced the Xperia XZ Premium with its Exmor RS sensor.
Some manufacturers themselves will even use DxOMark scores as a selling point when launching their products. Companies such as HTC or even more recently Google have been commended for their devices' excellent DxOMark scores. Ultimately, this is where the big question of DxOMark's objectivity concerning their scores can arise.
If DxO advises companies, is it an objective judge of quality when ultimately these scores are the result of their advice? Is it appropriate for a company to be both judge and jury? Doubts are understandable, especially where scores such as these can be viewed as being fairly subjective. The blurred relationship between certain manufacturers and DxO could be looked at as being more than a little suspect, especially for those who love a good conspiracy story.
However, the seniority and reputation of the brand in the field of image quality are strong enough arguments to attest to their good faith. At MWC, I was able to access to an example of a full test report and this was a very clear indication that the firm takes its business very seriously. The credibility of the entire company rests on its DxOMark scores, so there is absolutely no point in wanting to "cheat" the system they themselves established.
The tests that have been carried out by the press also confirms the importance of the DxOMark scores. When the Google Pixel appeared on their website with the best recorded score for any smartphone, many were skeptical due to the device's lack of an optical stabilizer and a dual photo sensor. The final results ultimately proved that the Pixel was rightfully one of the best camera phones on the market.
One tool, no more and no less
In the end, it's all just a question of perspective and analysis. Any kind of ranking or scoring system should ideally be approached with a degree of caution. Ultimately, it's the confrontation between different results, different feelings that makes it possible for us to get a better idea about a device's true potential.
DxOMark scores are an accurate (and reliable) tool for analyzing the performance of certain devices (including image quality), but this should not be the only way you should judge a smartphone's photo experience (just as an example, DxOMark doesn't test camera apps).
Did you already know about DxOMark scores? How big of an influence are these scores when you're looking to buy a new smartphone? Let us know in the comments below.