For Apple, "AI" apparently doesn't stand for "Artificial Intelligence," at least not in public. During today's WWDC keynote, the company never once said "Artificial Intelligence" or "AI." Instead, it announced Apple Intelligence, a new personal intelligence system built on generative models (Apple doesn't call them generative AI). It aims to integrate AI features deeply into the operating systems, including image generation, document summarization, and a major overhaul of Siri.
Apple Intelligence: Combining Personal Context with Generative AI
It was an open secret that Apple would announce AI features in a big way at WWDC, so the news itself was no surprise. The features themselves also don't go much beyond what OpenAI, Microsoft, and Google already offer, so little of it feels new. Still, the presentation was skillfully done, and the glimpse of how useful it could be to have these capabilities integrated into the OS on Apple devices is undeniably exciting.
"We're thrilled to add a new chapter to Apple innovation. Apple Intelligence will transform what users can do with our products, and what our products can do for our users. Our unique approach combines generative AI with a user's personal context to deliver truly helpful intelligence. And it can access that information in a completely private and secure way to help users do the things that matter most to them. This is AI as only Apple can deliver it, and we can't wait for users to experience what it can do," said Apple CEO Tim Cook.
Apple Intelligence, the personal intelligence system Apple announced, is a comprehensive set of features that works across iOS, iPadOS, and macOS. A beta will begin this summer, but it will be available only to English-language users in the US. Support for other languages, including Japanese, will wait until 2025.
Apple's emphasis has always been privacy, and it insists on handling most generative AI processing on the device itself. As a result, using Apple Intelligence requires considerable processing power: for iPhone, currently only the iPhone 15 Pro and iPhone 15 Pro Max qualify, while Macs and iPads need an M1 or later.
For tasks that can't be handled on the device, part of the processing is sent to the Private Cloud Compute platform, which runs on privacy-focused Apple silicon, and the results are returned to the device. According to Apple, the data on these special servers isn't shared externally, and even Apple cannot see it.
The new features enabled by Apple Intelligence are as follows.
A Major Siri Update
Siri is the personal assistant Apple released 13 years ago. People were amazed by its novelty at the time, but it saw little progress afterward and was often labeled "useless" compared with its rivals. With Apple Intelligence, however, Siri is being substantially reborn. Siri can now accept voice or text input, remember earlier queries, and respond intelligently based on them.
Siri's speech understanding has also improved dramatically. Even if you stumble over words, pause, or need to rethink your request midway through a query, Siri can process the information without trouble. Siri also has an "on-screen awareness" feature and can take in information from text. For example, if a friend shares an address in a message, you can simply say, "Add the address from so-and-so's text to his contact," and it will be added to your contacts.
Siri's actions also extend to images in the Photos app. In Apple's presentation, the user told Siri, "Show me photos of Stacey in New York wearing a pink coat," and the results appeared. The user then told Siri to "Add this photo to Stacey's profile note," and Siri understood the context and carried out the instruction accordingly.
These capabilities extend to all of Apple's first-party applications, as well as third-party applications that use the App Intents API. Siri can help you find specific information even if you don't remember whether it came from a text, an email, or an earlier note. For example, if you need to enter your driver's license number on a form, Siri can search the images on your device for your license and fill in the number in the required text field.
In another demo, a user asked, "When is my mom's flight landing?" Siri looked up the flight details and cross-referenced them with real-time flight tracking to work out the exact arrival time. When the user then asked about lunch plans with their mother, the plans appeared, and when they finally asked how long it would take to get from the airport to the restaurant, Siri plotted the route in Apple Maps.
Apple Intelligence Integrated into Apps
Besides text generation, image generation is perhaps the best-known use of generative AI. Apple has turned it into Genmoji (a portmanteau of "generate" and "emoji"), a feature for creating custom emoji.
Users give instructions through a text prompt. For example, entering "smiling face relaxing with cucumbers on its eyes" produces a Genmoji that adds custom decoration to the familiar yellow-faced emoji, which can then be used in messages. Several Genmoji appear to be generated at once, and users can swipe through the variations to find the one that best suits them.
Apple is also introducing Image Playground, which can be accessed through the Messages app. Using text or voice prompts, it can generate images in "sketch," "illustration," and "animation" styles. It can create images not only from your own photos but also from the profile pictures of people in your contacts.
Image Playground is also available as a standalone app, and third-party apps can use its API.
Features for summarizing long emails, rewriting text, and changing its tone are also being added. These are available system-wide across all of Apple's operating systems.
Incidentally, all of these AI features are powered by Apple's own in-house AI models.
In addition, as described below, Apple also appears to be offering the option of using external AI models such as those from OpenAI and Google when users want more advanced answers.
Partnership with OpenAI
OpenAI will be the first external partner for Apple's generative AI.
A basic version of ChatGPT powered by GPT-4o will be available for free in iOS 18 starting this fall and will be integrated directly into some apps. According to Apple, user inputs will not be stored.
Subscribers to paid plans such as ChatGPT Plus can also access ChatGPT's premium features, including image generation with tools such as DALL-E.
According to OpenAI, Siri can also tap into ChatGPT's intelligence when needed.
Apple is also considering further partnerships with other AI companies. Google Gemini and other models could follow as alternatives to ChatGPT, and talks with Google are reportedly under way.
Sources