Apple Intelligence gets smarter: Live translation, genmoji updates, visual intelligence, new dev tools and more

Apple has unveiled an ambitious expansion of its artificial intelligence capabilities, enhancing its suite of tools across its devices, including iPhones, iPads, Macs, Apple Watches, and Apple Vision Pro. Key innovations include Live Translation, advanced visual intelligence, personalized Genmoji, and the introduction of the Image Playground for custom image creation. Craig Federighi, Apple’s senior vice president of Software Engineering, emphasized the company's commitment to user privacy while making AI more accessible. "Last year marked the beginning of our journey to provide users with helpful, relevant intelligence that is easy to use and respects their privacy," he stated. With this update, the models supporting Apple Intelligence are becoming more powerful and efficient, with broader integration across all operating systems. One of the standout features is Live Translation, which allows users to communicate in different languages seamlessly during messaging and calls. This real-time translation capability operates entirely offline, ensuring that user data remains private. In the creative realm, the updated Genmoji feature lets users combine emojis and add text prompts for greater customization. Users can modify attributes to make these digital representations more closely resemble their friends and family. Additionally, the Image Playground introduces a ChatGPT-like generation system, enabling users to select various artistic styles or define their own for custom images, with strict user permission for data sharing. Furthermore, Apple’s Visual Intelligence enhances the user experience by providing contextual understanding of on-screen content. By simply tapping a button, users can ask for product identifications or information extraction, transforming it into actionable calendar entries or reminders through compatible applications. The Apple Watch also gets smarter with the introduction of Workout Buddy, a feature that provides real-time motivation based on user fitness data. This assistant delivers personalized feedback based on various metrics while maintaining user privacy by processing data locally. In a significant move towards developer engagement, Apple is granting access to its on-device foundation model through the new Foundation Models framework. This allows developers to incorporate privacy-focused AI features without relying on cloud connectivity. Applications can now implement functionalities like guided generation and natural language understanding with minimal coding requirements. Enhancements extend to the Shortcuts app and Siri, with improvements enabling more complex queries and contextual conversations. For example, students can now compare lecture notes with transcriptions through Siri without compromising their data's privacy. Additionally, users will see everyday enhancements across the Apple ecosystem, such as automatic categorization of reminders, order tracking summaries in Apple Wallet, AI-generated suggestions in Messages, and advanced summarization features in Mail and Photos. Apple continues to prioritize user privacy in its AI strategy, ensuring most processes occur on-device, with cloud computing only utilized when necessary. The company is also expanding language support, planning to add eight new languages by the end of 2025, with current support for multiple languages already available. Developers can access these features through the Apple Developer Program, with a public beta scheduled for next month. The full rollout is anticipated this autumn, compatible with the latest iPhone models and newer iPads and Macs.