- Expressive subtitles add feelings and (gasp) emotions to Android live subtitles
- Google Lookout gets new Gemini improvements
- Gemini on Pixel phones will now remember more information about you and your interests
Google has released a number of new AI features for most Android phones, as well as a few that only work on its Pixel line of devices.
Some of the new features work in Google GeminiGoogle’s AI app, but most of them work on the entire Android system. operating systemwhich means there will be big changes for Android users. Let’s break it down.
Expressive captions
First of all, there are expressive subtitles, which are a way to bring more emotion into the rather boring and lifeless world of subtitles and captions. Google presented Live subtitles on Android in 2019, so if something plays audio on your phone, it also has subtitles. They work in much the same way as automatic subtitles on YouTube videos – your phone generates them live, based on how it interprets what the person is saying.
The live subtitles remain rather lifeless, however. With expressive captions, Google uses the power of AI to help you better understand the context of what’s happening. You’ll see bracketed comments like (whispers) and (cheers and applause) appear among the captions, allowing you to better understand the feelings and emotions being conveyed.
Intense vocalizations will be written in (ALL CAPS), while vocalizations like (sigh), (groan), and (panting) will also appear. Ambient background noises will also be labeled. Since live captions are part of the operating system, you’ll now get expressive captions with most things you watch, so pay attention to that.
Gemini in gazebo
Google Search is an assisted vision application available on Android devices via the Play store. It uses AI to help people with low vision or vision loss explore the world around them using their phone’s camera. You can use it to read text, scan food labels, identify currency, and even scan your surroundings to identify things like doors and windows and know how far away they are.
Now Gemini AI is part of the Lookout app, which means it offers much richer and more useful descriptions. If you take a photo in the app, you’ll now hear its description read aloud in a natural voice and you can ask Loopout follow-up questions about it.
New Gemini extensions
The more extensions Google Gemini has, the more applications it can bring its AI magic to. Gemini now has a new Spotify extension, which means you can listen to your favorite songs or discover playlists from Gemini.
Gemini will now also connect to your phone and messaging app, meaning you can call contacts and send messages from Gemini. Google says that “soon” you will be able to use Gemini to control your smart home devices. Finally, Gemini will now also connect to Maps, which means you’ll get more information about locations.
More Android Features
Also in the December update, you’ll find new pizza-themed sticker combinations that you can share with your friends using Emoji Kitchen in Gboard, Google’s keyboard app. You can also use QR codes in Quick Share, making it easy to share photos, videos, and documents without having to add people as contacts or verify devices. All you have to do is generate a QR code that people can scan. Documents you scan using your phone will also be enhanced, with better contrast and white balance.
Pixel-only features
If you own a Google Pixel phone, you’ll get some additional features in the December Android update. There are many small updates to things like camera and security settings, but the biggest feature is the information Gemini records. This means you can ask Gemini to remember more about you, what interests you, and what you like, so it can tailor its responses to more of what will interest you.
Pixel users will also receive updates to the exclusive Pixel app, Pixel Screenshots. Pixel Screenshots uses Gemini to analyze and recall the information embedded in your screenshots. You can now use Pixel screenshots to save items when using Circle to Search with just one click.