Not only does Huawei’s Mate 10 models have a completely new design, its insides also contain something different not yet being used to its full potential. Huawei’s Neural Processing Unit (NPU) will especially shine when used for augmented reality apps considering its large number of incoming parallel inputs, and will also support artificial intelligence on the device itself i.e. offline as well.
Qualcomm no longer wants us to use the term ‘processor’, and this term would also be unsuitable for Huawei’s new chipset generation—the Kirin 970. This new core component increases the chipset’s complexity and prepares it for new tasks. That means artificial intelligence will be calculated more efficiently on the smartphone. And this is where the new NPU comes in.
What’s so special about what the NPU does?
This new sub-unit will perform tasks such as image recognition up to 25 times faster than the CPU—and consume 50 times less energy while doing so. This makes the NPU faster and more efficient than the GPU, which would bring four times the performance with eight times the efficiency according to Huawei.
But anything that sounds quite powerful must also have practical use cases and to that end, Huawei is still working diligently on additional software implementations. A few of them will be widely available shortly in the new Huawei Mate 10 Pro.
Image recognition: Use NPU and training data for that perfect food porn shot
If you take a look around at a restaurant, you’ll often see foodies taking pictures of their food and immediately sharing it online with friends or family. To make food look even more scrumptious, the image should look sharp, clear and beautifully saturated with color. But if the typical restaurant lighting conditions are not right, you can still make a last-ditch effort to save it with Instagram filters.
Inside the Mate 10 and Mate 10 Pro, Huawei uses a somewhat larger aperture, but otherwise hardware that is very similar to what we have already seen in the Mate 9. The difference is now it is boosted by the NPU. Huawei has gathered 100 million images to form a database with its artificial intelligence. The NPU detected known patterns and hardwired the training results into the new smartphones. Using this ‘training’ with its virtual mind, the NPU in the device can detect in real time what the searcher is actually capturing.
People, animals, nature, architecture, or spaghetti bolognese: The Kirin 970 detects subjects even without sending an inquiry to a home server, and uses this knowledge to set the right lighting values to process it accordingly. Skin can be smoothened, pimples are removed, everything behind faces or flowers is blurred. The red in strawberries is truly accentuated. You get the idea. The NPU can obtain more information from the raw material.
Exceptionally complicated lighting conditions like streets at night with powerful artificial lighting are the bane of current cameras. Thanks to the NPU, the optics capture the scene, mix different lighting and finds a compromise between sharpness and brightness. As a result, dark and bright areas are depicted with greater details than before.
Offline translations: Text only at first, but speech will come soon afterwards
Together with Microsoft, Huawei has implemented some prolific offline translation features. You can download 40 language packs that you can use to download text blocks. Text can either be copied from other apps, typed in or you can scan it from your surroundings with your camera. As a result, you can read Chinese street names, decipher bus lines or read the menu at a restaurant. Of course, it would also work without NPU, but the system works three times as fast with neural support.
Unfortunately, translation is limited to text only, although it’s only a matter of time until it works with spoken speech as well. Then the translator can pronounce the phrase for you, so that you can also properly understand the native speakers of the language in question. Looks like a competitor to the Pixel Buds is also coming.
NPU takes the burden off the CPU
Apart from the applications optimized for it, the NPU’s performance is currently underused, which is a shame because many apps could take advantage of it. Not only would they be faster, but their battery consumption would also be significantly reduced. What Huawei needs here is support from the wider ecosystem, and this is where Google’s machine learning framework effort, TensorFlow Lite, should be taken advantage of.
Its optimizations will make their way into many apps by the time Android P comes around. Huawei’s NPU will then recognize those apps and accelerate them accordingly by 10 to 100 percent. App developers would not need to make any additional optimizations.
Enthusiastic developers can in theory, produce a ten-fold acceleration. But this would require additional adjustments with Huawei’s proprietary HiAI in addition to machine learning optimizations (with TensorFlow or Caffe). Huawei estimates that the required work would occupy an employee for a week. It remains to be seen whether HiAI is of use outside of Huawei’s domestic market, where it operates its own app store.
The NPU is extraordinarily good at detecting patterns, although the training data is often in the cloud only. The interplay between cloud and offline AI is not yet balanced well, and we will only truly feel the benefits when the ecosystem has adapted to the new hardware.
Voice and handwriting recognition are beneficial in translation, image detection helps in automatically tagging and finding photos later, and distinguishing voices from background noises is useful in phone calls—the latter is already available in the Mate 10 Pro.
Many tasks that until now seem quite complex and for which dedicated hardware had to be used, such as hearing aids, can now be accomplished with properly equipped smartphone and an app, and the best part of all is that they no longer have to query a central server for these tricks.
What do you think of the Mate 10 models' AI functions? Will you increasingly focus on AI features when deciding on your next purchase?