How we test smartphones?

Mobile phones, especially smartphones, have become our essential pocket companion that keeps us connected, informed and entertained. A veritable pocket PC loaded with so many features and capabilities, how to determine if a phone can live up to its marketing hype and perform reliably?

At MingJian we take this questions seriously and spare no expense to help ensure consumers get the best value for their money. We participate in the largest and most rigorous test of mobile phones globally, organized by International Consumer Research & Testing (ICRT), together with consumer organizations in more than 10 countries and carried out in a top electronics labs in Europe. Every phone is subjected to an identical battery of more than 500 tests, with more than 1,000 models of mobile phones tested to-date. All the products tested are purchased by anonymously by independent consumers, to ensure the authenticity of the test.  

The test criteria is regularly updated to match consumers’ consistently changing requirements and newly introduced features and capabilities. Our test criteria is divided into the 9 most important aspects for consumers: camera and video, calling, internet, music, battery, ease of use, display, navigation and durability.

Camera

For the camera video test, we assess both camera and video, including sound and image, quality.

(camers test)

  • Camera quality: usually mobile phones have front/rear two cameras, and we carry out test for both cameras. The main rear camera receives the most weighting with comprehensive tests of standard viewing, low light viewing, resolution, color reproduction, distortion, image stabilization, dynamic range, etc. We assess the camera versatility, assessing each of the features and different modes. We additionally assess the response speed, both of the shutter lag as well as start-up time to capture timeless moments.
  • Video: this test includes both image recording quality and audio recording quality. For video image recording quality, we test in a variety of environments from daylight, fast-moving action to lowlight indoor and night tests, assessing the image quality on a reference PC monitor. We assess the image stabilizer feature to assess how well the mobile phone deals with shaky hands.We then assess the audio quality test of recorded videos using a pair of reference headphones.

Calling

As a phone, it is essential you can clearly understand the caller and that they can clearly understand you. Further, handset sensitivity is important to ensure that the call reaches you and that the connection is stable and not dropped.

 

 

(Calling sound quality test)

For Calling sound quality we assess the speech quality both making and receiving phone calls both in a quiet setting without ambient noise as well as simulating a noisy street environment and café with background music. In the handset sensitivity test, we gradually reduce the signal strength to see where the call drops out. We also assess the signal reception signature across the entire device to identify any areas of weak signal that may result in a dropped call if covered by the user’s hand.

(Signal sensitivity test)

Internet and Email

As for searching tools for information, the browsers on mobile phones are one aspect of the mobile phone test as well as the convenience to send and receive email.

  • Surf speed: we compare the download/upload speed which manufacturer claim and test download/upload speed (peak data rates) via real phone network.
  • Web browser convenience: this aspect mainly tests the usability of web browser and entering bookmark, as well as whether the browser can save bookmarks and browsing history, etc..
  • Email: we assess the convenience to set-up various email accounts, compose, send and receive email.

Music

Many of us use our phones as our primary music player when on the go. Our music test evaluates the music sound quality both through headphones as well as the integrated speakers.

  • Music sound quality: we use every mobile phone to play different kinds of music (such as pop, classic), and test the sound quality by using the delivered headphones as well as high quality reference headphones. We test the maximum loudness to ensure that maximum decibel levels are not exceeded that could cause long-term hearing loss. We also assess the ease of music transfer and the versititly for the different music codex the mobile phone can play.
  • Sound via integrated speakers: To share music or a video with friends, the integrated speakers are important when there is not an external pair of speakers available. We assess the overall sound quality via integrated speakers, as well as  the position of speakers for the stereo sound imaging.

Battery life

A battery that can last through the day without recharging, despite heavy use, is a basic consumer requirement, yet one which not all phones can deliver on.

This test consists of two aspects: battery life and handset charging. For battery life, we charge the battery up to full and then see how long it can last making continuous calls. We then charge it up to full again and test how long it can last browsing the internet on a 3G connection.

The charging test assesses the battery charging level after 15 minutes from empty and how long it will take to get full charge.

Ease of use

Who has the time to read through a complex user manual or struggle to make sense of a complicated menu structure? Consumers want mobile phones loaded with essential features, designed with usability in mind that are intuitive easy to use. This test is consist of 3 aspects: ease of use for basic functions, handset portability and handset capabilities.

  • Ease of use for basic functions: we assess the user experience of text input, touchscreen feedback, device layout and other general functions (like navigation menu/setting, starting time, etc.).  
  • Handset portability: How well does the handset fit into the pocket for easy portability? Is it comfortable or does it have any sharp edges?
  • Handset capability: the features and specifics of every mobile phones are assessed.
  1. Connections and versatility: we test the handset GSM bands and data transfer, as well as what kinds of interfaces one mobile phone has, such as bluetooth, USB, GPS receiver, NFC, memory card and so on. In addition, we also test the handset versatility, including WLAN tethering possible, compass, MP3 player, FM radio, acceleration sensor, and so on.
  2. Capacity: it mainly tests how many available storage capacity and maximum memory one mobile phone has, and whether it is possible to install a big game, remove one small app possible and so on.
  3. Synchronization: we test the performance of synchronization versatility, by synchronizing music (via USB), pictures and videos onto mobile phones without any addition software or internal shifting procedures. Besides, we also test the performance of synchronization files from mobile phones back up to computer.

Display

How clear and bright is the screen? Using an action movie, we test the smoothness of motion shots as well as the accuracy of colors.

We then test the phone in various light conditions to assess the readability of the screen from the side with glare and in bright sunlight.

Navigation suitability (out of the box)

Mobile phones are increasingly replacing built-in and standalone GPS devices for navigation. Yet how accurate are they and how well do they get you from point A to point B? Our test include aspects of expert GPS quality test, first GPS fix, sound loudness GPS, accuracy of positioning and general performance test.

In the test of expert GPS quality, our expert test GPS performance in different conditions: GPS offboard without voice guide, GPS offboard with voice, and onboard app.

Handset Durability

To assess if the phone can live up to a rough and tumble daily life, we check the build quality of every mobile phone, and carry out tumble test, scratch resistance test as well as water resistance test.

In the tumble test, we put each mobile phone into a rotating drum, simulating a fall from a height of 80cm, repeated 50 and 100 times to assess the drop resistance of both display and body.

In the water resistance test, we put each mobile phone under a rain shower for 5 minutes and immediately check whether the phone can be used as normal. We then let the phone dry out and and 3 days later check the functions once again.

In the scratch test, we use a calibrated tool with different amount of force to scratch the display and body of every mobile phone, to see their ability to resist scratch.

(Rain test)

Weighting

According to the importance of different test, we assign weights for every test as follows:

  • Camera ...........................25%
  • Display.............................20%
  • Performance………......17.5%
  • Music.............................. 10%
  • Battery............................7.5%
  • Calling...............................5%
  • Durability...........................5%
  • Ease of use.......................5%
  • Navigation ........................5%

Go to mingjian mobile phone test, to check all of the latest smartphone reviews.