A Samsung Galaxy S25 Ultra smartphone at a pre-Galaxy Unballaged media event in New York, United States, … [+]
Along with news that Gemini replaced Bixby on Samsung phones this week, Google updated Gemini’s capabilities on millions of Android phones.
Gemini can now perform tasks across multiple apps with a single prompt. For example, I asked the AI service to find Caribbean restaurants in North London and send them to my friend on WhatsApp, which he did. I also asked him to find out when Mobb Deep is playing in London next and add it to my calendar, which he finished.
This new multimodal feature works in conjunction with Gemini’s app extensions. A handful of Google apps (and now select Samsung apps) power the AI responses, including workspace apps, Spotify, Messages, and WhatsApp.
This is an important step in making AI assistants more like real-life assistants. What’s been missing throughout the Alexa, Siri, and Google Assistant era is the flaw to connect to other services and perform simple tasks. You’d imagine that asking a Google Nest speaker to add ingredients to a shopping list would be easy, but it’s this type of functionality that has eluded these products for years.
This has meant that aides opposed to the vote have failed to live up to the promise or hype. Multimodal skill adds a layer of intelligence to an uneven experience, but it’s still limited to selecting apps and that can confuse the boundaries of the new feature.
The key to using the feature is to remember that it only works with Google and Samsung apps. Gemini found my work task list in Google Docs and shared it with my partner as requested. But it won’t send a message on your behalf, so I couldn’t get it to text my hairdresser to make an appointment and add that to my calendar.
Additionally, the Google example gives how the automation works, asking it for restaurant recommendations and then sending the list to a friend, removes a key part of using Google Maps: Quality Control. I want to check establishments before sending them to someone, who still needs a human touch.
Spread Gemini far and wide
Elsewhere, Gemini Live – the conversational part of the AI service that allows instant back-and-forth chat – now supports uploaded images, files and YouTube videos. For example, you can upload a photo of your fridge contents during a chat with Gemini Live about what recipes to cook that week. This is only available on Pixel 9, Galaxy S24 and S25 phones.
Circle to Search has also been updated with AI previews. The search feature, launched last year, allows users to draw a circle on images to perform an image search. Now, Google’s AI summaries that appear on top of Google searches will also appear in a circle to search results, although the accuracy of the previews has been criticized since its launch last May.
With the exception of Gemini Live updates, interestingly, Google is adding next-generation AI experiences to millions of smartphones without requiring new hardware or a subscription.
Google says the new Gemini multimodal feature is available to all Gemini users on web, Android, and iOS. Google has a paid AI subscription tier and Samsung has teased Galaxy AI charging at the end of this year. For now, however, these AI updates are free and available to most smartphone users.
