Sensor shift stabilisation, new RGBW sensors, personalised AI algorithms, and more – here’s why your selfies and holiday photos are going to level up next year
Smartphone cameras are no longer just the most convenient option – they are quickly becoming the best one, beating traditional digital cameras with computational power and innovation.
In 2021, smartphones with multiple lens systems became mainstream. Improved sensor size and resolution mean users can capture more details and render images accurately in complex lighting conditions. Artificial intelligence (AI) algorithms enhanced many facets of image processing. And they’re not done. A slew of changes are coming in 2022, as smartphone manufacturers push the envelope.
What are the trends that will drive this change? At a recent webinar, Global Mobile Camera Trends 2022, organised by Counterpoint Research, experts from TECNO, Samsung, and DXOMARK Image Labs, discussed some of the key trends we are likely to see in the coming year – and what we will soon be holding in our hands.
Dual OIS: Staying steady
Not getting the sharp images you wanted of your vacation views? Blurry and shaky photos are a common experience with most smartphone users, and are especially pronounced in low-light conditions as the shutter stays open longer to capture more light.
Smartphone manufacturers counter this issue through Optical Image Stabilisation (OIS) technology, where the lens "floats" inside the camera assembly. The OIS system stabilises the image or video by "shifting" the lens to counter the shakes.
OIS technology is improving every year, with larger degrees of movement being taken on by the camera stabiliser. According to Jimmy Hsu, senior product manager, image technology, TECNO, pre-2019 the trend was for OIS to be about one degree of stabilisation. According to Hsu, in 2021, standard stabilisation has increased to three degrees thanks to pan-tilt OIS.
In 2022, the company aims to introduce the first Android phone with sensor shift. “Compared to conventional Optical Imaging Stabilisation, TECNO’s sensor shift technology can achieve a response frequency of 5,000 adjustments per second,” Hsu said at the webinar. By combining lens shifting seen in conventional OIS systems and the newer sensor shift technology, TECNO is setting its sights on stabilisation up to five degrees.
High-resolution sensors with pixel binning: Balancing resolution and light sensitivity
Presenting trends in camera resolution, Tarun Pathak, director of research, smartphone, Counterpoint Research, said about 40% of all smartphones in Q2 of 2021 had 48-64 megapixel (MP) sensors, and 3% of all smartphones had 108 MP sensors. According to Pathak, smartphones with 200 MP sensors are likely to launch in 2022.
Sensors with a high pixel count tend to capture more details from the scene. However, the size of pixels on such sensors is usually smaller. “The smaller the pixel, the less light you can capture in each pixel, and the harder it is to get a good image quality in darker environments,” said Pan Xuebao, VP and head of R&D, Samsung Electronics.
However, sensor manufacturers such as Samsung are deploying an algorithm-driven technique called pixel binning that enables high-resolution sensors (with a small pixel size) to perform better in low-light conditions. “Just like our human eye will dilate when we are in a darker environment to let in more light, similarly, in low light conditions, our algorithm will automatically put the sensor in high sensitivity mode,” Xuebaosaid. In this mode, the smaller individual pixels ‘combine’ and act like bigger pixels. This mode is turned off in brighter conditions.
Continuous optical zoom: Closer to the action
Phones are getting sleeker, but these aesthetics can be a technological challenge. This trend is presenting more of a challenge for manufacturers as they try to incorporate bulky telephoto lenses. Such technologies, such as periscope lenses which use innovative positioning to allow for 5x or 10x optical zoom, started appearing in smartphones during 2021.
The next step is to offer continuous optical zoom, which will allow users to shift between different focal lengths smoothly. This requires precise movement of lenses within small spaces without affecting image quality.
At the webinar, Hsu predicted the market could see continuous optical zoom in the second half of 2022, allowing users to shift between 3x to 5x optical zoom. Driven by advances such as long-stroke motor development and Active Optical Alignment, we could even see 3x to 10x zoom in 2023, he said.
RGBW Sensors: Let there be (more) light
Cat photos not up to the mark? It could be down to the lighting. Indoor environments have less than ideal light conditions. Yet, that’s where people usually take most of their photographs.
“[A solution to] low light is the holy grail of mobile photography,” said Hervé Macudzinski, image science director and product owner of DXOMARK Image Labs, which tests camera, audio, display and battery quality. “This is one [aspect] that can lead to disappointing quality. In low light, users capture portrait photos, pictures of family, pets and so on. So there is a lot of challenge in this use case.”
While smartphone manufacturers use different techniques to enhance low-light performance, RGBW technology addresses the issue at the sensor level.
The camera sensor captures colour information through a mechanism called the Colour Filter Array (CFA). Traditional camera sensors use CFAs arranged in the Bayer format, which comprises a pattern of Red (R), Green (G), and Blue (B) filters. RGBW sensors add White (W) filters to the mix.
According to Hsu, adding white pixels increases light sensitivity by 60%, which means the sensor should be able to absorb more light. Smartphone manufacturers are also developing Glass+Plastic lens technology - many smartphone lenses are constructed out of plastic, but introducing glass elements into the lens construction improves transmission of light. “With the combination of both, the result is an overall 200% increase of light-intake,” Hsu said.
The low-light performance of smartphones looks set to improve as these two technologies appear on smartphones in 2022.
Localised algorithms: Using AI to personaliseportraits
As cameras improve, we can see ourselves more clearly. New technologies allow our portraits to be more accurate, especially for people of colour.
While hardware enhancements are essential, smartphone software is playing a more significant role in determining the final image or video quality of these portraits. “We need to have very good processing – that is, to understand what the user’s intention was and modify the image adequately,” Macudzinskisaid.
That’s why some smartphone manufacturers use big data and deep neural networks to drive personalised and localised image processing. For instance, TECNO’s TAIVOS signal processing technology recognises and accurately reproduces dark skin elements. According to Hsu, the company uses a large dark skin database to “produce more than 6,120 Evaluation Scenarios spanning 115 countries”.
For portrait photography, the AI engine recognises and classifies different elements such as faces and hair colour to determine the subject's age, gender, and skin tone. The photo is then optimisedfor better clarity, colour, and light, which helps with more accurate reproduction of darker skin tones – even in low-light environments.
Scene, light, and colour enhancement: Behind every subject is a background
It’s not just the subject of a photo that matters – it's what’s behind them. Whether it’s the sun or a beautiful view, smartphone cameras are adapting to capturing these details better.
With traditional photography, professional photographers use their experience and training to make adjustments that highlight not just the subject but also background scene elements. They also typically spend significant time in post-production that brings images to life.
“Smartphone cameras need to provide all these capabilities in the default mode,” Macudzinski said. “That means the final smartphone user does not have to think through everything. They just have to press the button.”
That’s where AI-driven image processing comes in. According to Counterpoint Research’s whitepaper, AI is becoming advanced enough to recognise dynamic scene elements such as trees or sky in the background. Once identified, the image-processing system optimises their colour, contrast, and exposure.
AI will also help interpret light information. For instance, if a user is shooting against the sun, the algorithm works to underexpose the sun and brighten the subject. Smartphone manufacturers are also working on software-based solutions to correct distortions caused by wide-angle lenses.
Time of flight and dynamic vision sensors: Getting a sense of depth
Photos with sharply rendered foreground and blurred background can make for a pleasing effect – especially while shooting portraits. However, to do this, the smartphone must recognise depth and subject edges.
With Time of Flight (ToF) and Dynamic Vision Sensors (DVS) improving, smartphones will get better at generating 3D models of the scene and detecting edges. This will allow for more precise measurements and offer users features such as. enhanced and gradual blurring of backgrounds.
“ToF and DVS could be the opportunities where mobile phone cameras start to outperform (DSLR) cameras,” Hsu said.
A brighter, more intelligent 2022
These trends point to technology growing more intricate and catering to diverse use cases – whether that’s through personalised portrait rendition, stable imaging or allowing users to zoom in to the action.
However, the advancements will not be defined just by cutting-edge features on pricey smartphones. It’s a win across budget tiers - the developments result not just in technological advancement, but also trickle-down premium features to smartphones in the low and medium price bands. With the way smartphone cameras are levelling up, the best moments of your life will be remembered more clearly than ever.
Looking ahead with TECNO
Hosted by Counterpoint Research, Global Mobile Camera Trends 2022 Innovation Talk discussed new trends in imaging technology. TECNO aims to provide an excellent photography and videography experience through constant innovation and a deep understanding of consumer needs.
Source: TECNO Philippines