Gemini’s new beta feature provides proactive responses based on your photos, emails, and more

On Wednesday, Google introduced an innovative beta feature within its Gemini app designed to enhance user experience by providing tailored responses. This new functionality allows the AI assistant to connect with various Google services, including Gmail, Photos, Search, and YouTube history, to deliver more contextual answers than ever before. Previously, Gemini could access information from these platforms, but the latest update enables it to reason across your data. For instance, it can link email threads to related videos you’ve watched, showcasing its ability to understand context intuitively. Dubbed Personal Intelligence, this feature is turned off by default, allowing users to decide if and when they wish to integrate their Google applications with Gemini. Google acknowledges that not every user is comfortable with AI accessing their photos or viewing their YouTube history. Should users opt to connect their apps, Gemini is programmed to use Personal Intelligence only when it deems it beneficial. Josh Woodward, Vice President of the Gemini app at Google Labs, emphasized the feature's dual strengths: the ability to navigate complex sources and retrieve specific details from various media to answer user queries effectively. In a practical example, Woodward recounted a scenario where he needed to remember his car's tire size while at a tire shop. While standard AI chatbots can provide general information, Gemini took it a step further by suggesting all-weather tires after recognizing family road trip photos in Google Photos. He also mentioned how Gemini retrieved his license plate number from an image stored in Photos. Woodward praised Gemini's ability to provide personalized recommendations for books, shows, clothing, and travel. He highlighted that it has been particularly useful in planning family trips, suggesting unique experiences rather than typical tourist attractions by analyzing previous interests and trips. To ensure user safety, Google has implemented safeguards around sensitive topics, stating that Gemini will refrain from making assumptions about delicate information such as health unless prompted by the user. Furthermore, it is important to note that Gemini does not train directly on personal Gmail or Google Photos content; rather, it utilizes specific prompts to enhance its responses without directly accessing user data. The rollout of Personal Intelligence is currently available to Google AI Pro and AI Ultra subscribers in the U.S., with plans for expansion into more countries and the free tier of Gemini. Users are encouraged to explore this feature with prompts like, "Help me plan my weekend in [city] based on my interests," or, "Recommend documentaries based on my recent curiosities."