Apple’s new AI image tool just dropped and for the oft-flashy tech leader, the open-source AI image editing service is remarkably subtle and low-key. The Apple AI image editing open source code was announced via a conference paper published at the International Conference on Learning Representations (ICLR) 2024. Instead of releasing another tiresome chatbot assistant who can do it all on your Apple devices, the company went into a very specific segment of the market that can still eventually be tied back into their Apple services—photo editing. Teaming up with researchers at the University of California at Santa Barbara, the Apple open-source image model was released as free to use and available online for those who wanted to set it up and try it out for themselves. The magnanimous move is much appreciated.
The Reasoning Behind Apple’s New AI Image Tool
AI image generation is not a new concept—there are a host of tools available online and they only seem to be getting more advanced with every iteration of releases and updates. A majority of the AI tools that are available on the market right now are capable of taking user inputs and generating something interesting but they aren’t very capable of refining the image to give you exactly what you want. In a basic sense, AI image generators are quite capable of understanding basic keywords and phrases in order to generate an image, but when put together, they appear to be quite lacking. Taking an existing image and refining it further is also not something that existing tools are quite as efficient at.
This is where Apple’s new AI image tool comes in. The open-source AI image editing model by Apple enhances the capabilities of instruction-based image editing without requiring you to whip out overly elaborate descriptions to make it happen. The study conducted by the company and associated researchers focused on enhancing what Multimodal Large Language Models (MLLMs) are able to process, resulting in what they call MLLM-Guided Image Editing (MGIE). To do this, the researchers reportedly evaluated “various aspects of Photoshop-style modification, global photo optimization, and local editing.”
Essentially, Apple’s AI image editing open-source model is capable of editing an image to suit your style without you having to spend energy balancing shadows and playing around with hues to get an image right. At the same time, it also provides higher-level editing features like adding and removing objects from your image with ease. How efficient is it? It’s not nearly good enough for you to cancel your Photoshop subscription, but a tool like this can easily find use in the Apple ecosystem. Refining the accurate interpretation of commands is where we need most energy focused and that is precisely what the MGIE model claims to do; it learns to understand concise instructions and provides guidance for image editing using latent imaginations that you don’t necessarily have to feed into it.
What Does Apple’s New AI Image Tool Do?
The Apple AI image editing open-source tool presents a method of altering an image based on written instructions. With a more qualitative interpretation of the image instruction, the MGIE AI tool provides you with a final image that is more accurately edited to match your desired image. The tool was pitted against other image diffusion models such as InsPix2Pix, and in every instance, it performed more accurately, efficiently interpreting the task at hand.
When instructed to “remove a christmas tree from the background” or to “let the donuts have strawberry glaze on them,” the MGIE model was the most efficient at following the instructions without altering the entire image unnecessarily. Similarly, when prompted to add a Star Wars background, the Apple open-source image model is able to connect Star Wars with the addition of details such as spaceships or light sabers. The tool is quite “smart” at interpreting and imagining the relation between words in a cue and that’s what makes it so efficient.
For those who know what to do with code, Apple’s open-source image model is available on GitHub for free, and coders are welcome to experiment with the AI model and see just how it can be put to use. A demo web version of Apple’s new image tool is also available on Hugging Face for immediate testing. Apple may incorporate a more refined version of this AI image editor in future updates and iterations of its products but for now, it seems satisfied to see where the researchers take their explorations.
Windows Central writes that an Apple AI tool for Microsoft would be a big step forward for Microsoft’s own AI pursuits that have quite significantly ramped up these last few months, and we’d have to agree. With the company’s apparent ambitions to absolutely dominate the AI race, an Apple AI tool for Microsoft could be a fairly reasonable collaboration if Apple actually turns the image generator into a revenue source later on. The Microsoft Copilot app only recently arrived on the AppStore a month ago, a tool built on the backs of GPT-4 and DALL-E 3. iPhone and iPad users have Siri as their personal assistant but can turn to the newly added app for their AI needs, which includes an image generator as well.
From our own testing, the images generated by Copilot aren’t the greatest around—they are very clearly AI-designed and often inaccurately designed. This is a good thing to dissuade our worries about image generators getting terrifyingly good, but not useful enough to be practically implemented. Incorporated with elements of Apple AI, it could be improved significantly. For now, we have Microsoft’s Copilot tool doing its own thing with refining the Microsoft 365 services and experiences, while Apple’s AI endeavors look for a direction to grow in.