Apple and OpenAI had reportedly been in discussions to integrate OpenAI's generative AI technology into Apple's OS, including iOS, but The Information now reports that an agreement between the two companies has finally been signed. In addition, it seems that Apple plans to use OpenAI's generative AI technology to significantly improve its voice assistant "Siri," which has not exactly been praised for its intelligence up to now.
The Apple-OpenAI Deal Was a "Long-Held Goal"
The Information reports on this major deal based on accounts from a person who heard about it directly from OpenAI CEO Sam Altman.
However, OpenAI finds itself in a complicated position. It has received a massive investment of $13 billion from Microsoft and collaborates with the company on numerous product development efforts. Regarding this deal with Apple, Microsoft CEO Satya Nadella is said to be concerned about how it might affect the existing relationship between OpenAI and Microsoft.
There are also opponents within Apple. John Giannandrea, the head of machine learning, reportedly opposes the appearance of large language model-driven chatbots or similar features in Apple's software and products.
Nevertheless, The Information states that this has been a "long-held goal" for Altman, and he has finally achieved it.
According to The Information, Altman and others met with employees of Apple's machine learning team in mid-2023. This meeting, which took place around the time of last year's WWDC, resulted in Apple being granted access to OpenAI's internal APIs for testing purposes.
It seems Apple used this opportunity to enhance Siri, enabling the voice assistant to handle more complex queries. The Information reports that by gaining access to a leading LLM, Siri acquired impressive new skills, such as understanding the context of user commands. Apple is reportedly exploring several ways to utilize ChatGPT within Siri, including a transition to a chatbot similar to ChatGPT itself.
Siri May Make It Easy to Voice-Control Existing Apps
Additionally, according to Bloomberg, the improved Siri will allow users to perform specific app functions—such as sending documents or emails—using their voice. As previously reported, this appears to be part of an internal project reported as "Project Greymatter".
It seems this assistant will be able to analyze a user's phone activity and automatically enable Siri-controlled features. According to additional information from Apple Insider, when generating replies or summaries, Siri will be able to take into account entities such as people and companies, calendar events, locations, and dates.
According to Bloomberg, it will support "hundreds" of commands, but initially it will only be able to process one command at a time. Later, Siri will be able to support multiple tasks within a single request.
Initially, supported commands will include sending or deleting emails, opening specific sites in Apple News, emailing web links, and requesting article summaries. Once multiple commands become possible, users will be able to summarize a recorded meeting and text it to a colleague. "Or, theoretically, you could crop a photo on your iPhone and email it to a friend," says Bloomberg's Mark Gurman.
It also appears that a notification summarization feature, internally called "Greymatter Catch Up," is being developed simultaneously. This feature is also linked with Siri, allowing users to request and receive a summary of recent notifications through the virtual assistant.
However, these features won't be arriving immediately. The new version of Siri won't be included in iOS 18, but may be released as an update early next year. Also, initially it seems it will only work with Apple's own apps.
It's unclear which AI model will be used for this, but given that the agreement with OpenAI has now been signed, it's likely that OpenAI's GPT-4o or similar models will be utilized.
Incidentally, regarding Microsoft's concerns about this partnership between Apple and OpenAI, CEO Nadella reportedly pointed to Microsoft's massive investment in OpenAI and demanded a share of OpenAI's future revenue. In other words, the spread of generative AI on Apple's platform could potentially benefit Microsoft as well.
Sources