Apple accelerates its genAI efforts with LLM image-editing tool

Apple is pushing into generative AI (genAI) in a big way with a new tool called Keyframer; it’s designed to give users the power to animate static images using text prompts.

Keyframer is described in a recently published Apple research paper and what it does seems impressive. To create animated illustrations from static 2D images, users just need to pop in an SVG image, provide some text prompts, and let Keyframer build CSS animation code to animate that original image. Then you can fine-tune the creation.

Apple’s AI arms race is intensifying

This is just the latest in an accelerating rash of AI-related releases from the company. Through clouds of fabled memory, some may recall expectations that Apple will make big moves in AI this year.

Since then, in stealthy silence, the company has begun letting slip numerous breakthroughs, and as the velocity of news intensifies it’s becoming clear that Apple is piling resources into the effort. This is an AI arms race, and Apple has no intention of being left behind.

Earlier this year, the company introduced an AI tool to perform pixel-level image editing — MLLM-Guided Image Editing (MGIE) — which makes it easy to perform image edits using text commands. The confluence of Apple technologies hints that this also means you’ll be able to use Siri to make those edit commands, (hopefully, with results at least as impressive as the James Fridman Twitter feed).

Research and development

Apple’s recent claimed acquisition of the iWork.ai domain has also provoked a lot of speculation concerning its plans to deploy genAI in its devices.

Apple’s machine learning teams have made 16 additional research papers/discussions available since the beginning of the year, with a view to LLM development, healthcare, speech and more.

Toward the end of 2023, Apple’s teams unveiled three highly significant technologies: the capacity to run Large Language Models (LLM)-based AI efficiently on devices; the rollout of the ML Explore machine learning framework for Apple Silicon; and the introduction of Ferret, which optimizes machine learning. The teams have also created a model that swiftly creates avatars from video.

Apple is also investing quite heavily in AI start-ups. One recent report claims it has its eyes on a German firm called brighter AI, which focuses on image privacy.

With 32 such purchases in the last year, the company seems to be acquiring AI firms at a rate to outpace others in the space. “There’s an AI arms race going on, and Apple is not going to be on the outside looking in,” Wedbush Securities analyst Daniel Ives said.

Intentional steps that get results

Apple being Apple, the targets of these purchases are hard to determine, but the company does appear to be intentional, focusing its efforts on particular domains such as machine image intelligence and AI.

(Earlier AI-related purchases also reflected this, with Voysis providing natural language voice assistants and WaveOne delivering intelligent video compression tools.)

We can easily imagine how the company seeks to deploy these technologies. Apple sees privacy as a human right, which means part of its promise will inevitably be to create highly useful tools that can do most of what they need to do on the device itself — hence, the importance of the company’s earlier research into edge AI processing.

All eyes on WWDC 2024

Apple CEO Tim Cook has gone on the record to promise Apple will share more details concerning its work in the space “later this year,” which most industry observers now expect means at WWDC 2024 in June for wide distribution with new operating systems in fall.

There’s a lot at stake. Counterpoint recently predicted that almost 1 billion GenAI equipped smartphones will ship between now and 2027.

If Apple wants to maintain the trajectory of iPhone ascendancy it will need to play in the genAI space. However, if Apple’s iPhone does become a genAI machine, the number of genAI-equipped smartphones sold in the coming couple of years may absolutely eclipse Counterpoint’s expectation. We’ll soon find out if Siri is coming along for the ride.

Please follow me on Mastodon, or join me in the AppleHolic’s bar & grill and Apple Discussions groups on MeWe.