With the P30 Pro, Huawei shot for the moon with a camera phone equipped with a Swiss army knife of lens and a new imaging sensor and Ai technology to capture objects far and wide, even in the dark. While the P30 Pro launch dazzled many with its ability to capture images of the moon, MingJian’s lab tests reveal that its imaging capabilities are limited by earthbound realities, exhibiting some significant flaws.
For a comprehensive objective evaluation of P30 Pro’s camera performance, samples were purchased anonymously at retail and sent to an independent specialist imaging laboratory in Europe where we test digital cameras together with Stiftung Warentest, Which? and other European consumer organizations. The P30 Pro was tested alongside five other flagship smartphones (Huawei Mate 20 Pro, Huaweit P20 Pro, Samsung Galaxy S10, Apple iPhone Xs, and Google Pixel 3 XL) as well as compacts, DSLRs and mirrorless system cameras.
Our digital camera test program differs from our mobile phone test, where the camera (photos and videos) assessment comprises 25% of the rating score. In the camera test program, image quality is of critical importance. A top-rated camera produces sharp, clear, undistorted images, with accurate colors, and low image noise free from artefacts.
How did P30 Pro perform against the other flagship smartphones in the MingJian camera lab test?
- Apple iPhone Xs 83
- Google Pixel 3 XL 83
- Huawei P20 Pro 82
- Huawei Mate 20 Pro1 65
- Samsung Galaxy S101 65
- Huawei P30 Pro1,2 61
Ratings on a 50 to 100 point scale with 100 the maximum possible score.
All cameras should exceed minimum thresholds for standard viewing tests, if score below the minimum threshold for a critical image assessment criteria, it triggers a delimiter, limiting the overall rating score.
1 Delimiter for extreme distortion of ultra-wide lens.
2 Delimiter for inaccurate color reproduction and high image noise in lowlight.
What accounts for the poor camera performance of the P30 Pro against the iPhone Xs and its own predecessor, the P20 Pro? It can be largely attributed to two design modifications between the P20 Pro and the P30 Pro: changing the main lens image sensor from a traditional red-green-blue (RGB) sensor to a red-yellow-blue (RYB) sensor, and the addition of an ultra-wide lens. While the use of red-yellow-blue sensor increases responsiveness in lowlight, unfortunately it comes at the price of an unnatural color shift, high luminance noise, and color noise. The ultra-wide lens exhibits severe image distortion, chromatic aberration, image noise and artefacts.
Similar distortion and other issues were found in MingJian lab tests of the ultra-wide angle lens of the Samsung Galaxy S10 and Huawei Mate 20 Pro, explaining their low scores
We evaluate lowlight image performance in a controlled lab lightbox scene with luminance level of 11 lux.
Zooming in on the upper right section of the scene above, compare the image taken by the Galaxy S10 against that taken by the P30 Pro. Comments from the lab: Galaxy S10 “normal colors, normal skin tones, a bit noise, good sharpness” vs the P30 Pro “colors shift into orange, skin tones shift into orange, disturbing noise, normal sharpness, strong artefacts.”
The difference in color accuracy is clearly visible comparing the skin tone of the mannequin and the side of the church between the two images. The color noise and artefacts of the P30 Pro image can be seen in the flecks of purple in the eye and on the lips of the mannequin, on the spires of the church, and in the leaves of the trees.
Zooming in on the checkerboard card and the yellow barge in the river, compare the accurate coloring of the image taken by the Galaxy S10 with the color shift to orange accompanied by purple luminance noise of the image taken by the P30 Pro. (Update 2019/06/03 – Please refer to notes at the bottom of the article for how to reduce the color shift and image noise with a firmware update).
Although the vast majority of smartphone users use the JPEG image as generated by their phone camera, we asked camera test program leader, Dr. Markus Bautsch from Stiftung Warentest in Germany if the luminance and color noise could be cancelled in post-image processing with an editing software like Photoshop? Dr. Bautsch responded “both kinds of noise can be cancelled, but at the expense of optical resolution and contrast. If you apply too much noise cancelling the image becomes too flat, and if you don’t apply noise cancelling you will see all these speckle patterns.”
While the ability to capture images in light levels lower than what is visible to the human eye, is an amusing novelty, most consumers are looking for a camera that can capture lowlight scenes as they see them with high resolution, accurate colors and low image noise. Due to the tradeoffs, a night vision feature would perhaps be better as a separate lens rather than as the main image sensor.
While the sound quality was excellent and among the best we have heard from a smartphone, unfortunately the issues of the RYB image sensor negatively impact the video image performance. Comments from Dr. Bautsch on the 4K video image quality “pale skin tones, (too orange, over-saturation), sharpening artefacts as well as texture loss or visible noise. The noise is a constraint clearly due to the large pixel number [40MP] of the main image sensor.”
Our lab tests have repeatedly shown that beyond a certain threshold, increasing the number of pixels does not guarantee higher quality images, and is often accompanied by increased image noise and artefacts. We have seen this not only on smartphones, but even in high-end system cameras. The highest rated camera we have tested to-date is the Nikon Z6, an enthusiast model with a 24 MP sensor, which has less image noise at the pixel level than its professional sibling, the Nikon Z7 equipped with a 45 MP sensor.
The benefit of an ultra-wide angle lens is its wide field of vision, able to capture a panoramic display with a single shot, without the usual pan-and-scan image stitching. However, this often comes with a tradeoff, namely image distortion.
Dr. Bautsch states: “In order to address distortion issues, all new mirrorless camera systems carry out automatic lens error compensation by default.” Unfortunately, the Huawei P30 Pro lacks distortion compensation, resulting in extreme image distortion and other issues clearly visible in images taken of a reference chart.
The grey edges of the reference chart in an undistorted image will be straight and align with the edges of the image frame, as seen in the image taken with the standard wide-angle lens of the iPhone Xs. However, in the image taken by the ultra-wide angle lens of the P30 Pro, the corners of the chart curve in, revealing full reference arrows in each corner, where only the tips should be visible.
As Dr. Bautsch points out: “Close observation of the images of the reference charts reveal that, in addition to extreme distortion, the images taken with the ultra-wide lens also suffer from color fringes (chromatic aberration), high image noise, and a lot of digital artefacts.” These issues are clearly visible zooming in on the lower left corner of the reference chart.
Compare image taken by the iPhone Xs against image taken by the P30 Pro below and notice the purplish light shadow next to the white arrows in the lower corner as well as along the arc of the half circle, this is chromatic aberration.
Further, compare the grass and pavement in the lower left section of the chart. The grass in image taken by the P30 Pro is blurred with a loss of detail, compared to image taken by the iPhone Xs. Also the pavement in image taken by the P30 Pro is an unnatural representation suffering from digital artefacts.
Unfortunately, the shortcomings of the RYB main image sensor and distortion issues of the ultra-wide lens hold back the performance of an otherwise impressive camera phone.
The telephoto lens is where the P30 Pro really shines. It had the highest resolution among the smartphone tested and, with a 5x native focal length, outperforms the 2x lens on the iPhone Xs and Galaxy S10 (the Pixel 3 XL is not equipped with a telephoto lens). Although the P30 Pro can digitally zoom to 50x, it is accompanied by a loss of resolution and increase in image noise, so best quality shots are taken at the native 5x telephoto setting. The detail and lack of blur in the telephoto images is assisted by the very effective image stabilization and Ai technology.
The image stabilization of the P30 Pro shows a marked improvement from the P20 Pro. The P30 Pro appears to apply on-device image stacking, taking several images and combining them to get the optimal image and reduce noise. Something frequently used by astronomers to correct for the rotation of the earth and movement of celestial objects, it is impressive to see this technique applied in a smartphone.
Ai scene optimization
The P30 Pro is breaking new ground with implementation of Ai to recognize different scenes and automatically enhance JPEG image rendering in-camera. This is just the beginning of a trend we are likely to see growing in the near future.
Overall, Huawei has incorporated a number of new imaging technologies in the P30 Pro previously unseen on smartphones, yet the implementation and integration is still a work-in-progress.
Comments from Huawei
We contacted Huawei asked to comment on the issues uncovered in our lab test. Regarding the image distortion issues, Huawei responded “image distortion is a common optical phenomenon found in ultra-wide lens. Image distortion correction technologies are something that we will continue to improve upon moving forward.”
Indeed, at the launch event for the Honor 20 Series in London, Huawei announced that ultra-wide lens on the Honor 20 and 20 Pro will be equipped with correction technology to reduce the amount of distortion.
Regarding inaccurate colors in photos and videos, Huawei indicated that “the latest OS update improves color accuracy,” and that “Huawei will continue to work on improving color accuracy in the future.”
***We recommend that all owners of the Huawei P30 Pro update their phone to the latest firmware to address the color shift and reduce image noise.***
After received the reponse from Huawei, the latest OS firmware update was installed on the device by the lab and images were taken to compare the impact. We can confirm that the colors after the firmware update are more natural, with more blue components reducing the previous shift to yellow and orange, as well as a reduction in color noise. The impacts of the firmware update are visible in the below before and after images of the mannequin’s face and hair. In the top image, before the firmware update, her face is yellow with purple flecks of color noise visible in her hair. In the bottom image, after the firmware update, the skin tone is more natural and the color noise has been significantly reduced.