Adobe is set to give new meaning to the act of photoshopping an image.
Starting this week, the company will begin adding features to the popular image-editing program that let users alter images with AI. The new generative fill tool can replace objects or backgrounds with imagery produced from a text prompt in a way that’s meant to blend seamlessly with the rest of the image.
For instance, as the Adobe video below shows, a user might select an area on a road in an image and fill in “yellow road lines,” switch out a forest background for a “wet alley at night,” or extend the dimensions of a photo with generated visuals that match its edges. In each case, the model would be capable of carrying over details like reflections, shadows, and textures.
Like any image-generation model of its ilk, the beta feature comes with certain behavioral quirks, according to John Metzger, director of product management at Photoshop. As with other image-generating AI, it can sometimes struggle to properly render hands and faces, though this can be mitigated to some extent by homing in on the body part in question and asking the tech to regenerate just that body part with extra layers that home in on the body part in question, Metzger said.
“It does great at lighting and shadowing and landscapes—it does great on the majority of things,” Metzger told Tech Brew. “It doesn’t do a great job with some animals because it doesn’t have enough in the dataset. If you try to generate a polar bear, sometimes it generates a white dog.” (He claimed this issue “will be fixed very, very soon.”)
As for fixing any body-part mutations one might encounter, “you just select the hand and you type ‘hand’ into the textbox, and it will regenerate another hand for you,” Metzger said.
Fending off upstarts: The rollout comes as Adobe has seen an influx of would-be competitors. Startups like Runway, Midjourney, and Stability AI are all geared around turning the latest advances in image-generating AI into creative software.
Keep up with the innovative tech transforming business
Tech Brew keeps business leaders up-to-date on the latest innovations, automation advances, policy shifts, and more, so they can make informed decisions about tech.
Adobe’s push is part of a larger generative AI road map called Project Firefly that it unveiled in March. It differentiates itself with an image generator trained only on Adobe Stock or public domain media—avoiding the iffy legal territory of other models trained on copyrighted material—and a focus on augmenting rather than generating content whole cloth, according to Ashley Still, the company’s SVP of digital media.
“There are certainly use cases where creative professionals want to just completely ideate, and they want to generate a complete image,” Still told Tech Brew. “But that is actually more in the minority, where really what creative professionals want to do is they have content…and they want to modify that.”
Adobe has also said it plans to eventually pay creators if their work is used to train AI, though details remain scarce on the terms of that program.
“Nutritional labels”: Adobe is also committed to marrying its push into generative AI with the industry group it leads to label media produced by generative AI upfront, the Content Authenticity Initiative (CAI), which claims around 1,000 members, including the New York Times, Nvidia, and Canon.
As part of that pledge, Photoshop adds digital content credentials to each image’s metadata that show how it was altered with AI, and images downloaded from Firefly automatically contain content credentials, Metzger said.
“We’re committed to making sure that as generative content is more and more common across Adobe’s portfolio, that we provide, effectively, nutrition labels for images,” Still said.
The new Photoshop feature is only the latest rollout under the banner of Adobe’s Project Firefly generative AI effort. The company also announced this month that it would be integrating the technology into Google Bard along with the CAI credential system.