Have you ever had a beautiful vision but lacked the drawing skills to get it down on paper? A new artificial intelligence (AI) system in pre-release from OpenAI has unlocked the artist in the machine. DALL-E, as the technology is called, can convert simple text prompts into digital illustrations in an array of styles, from the painterly to the photo-realistic — such as a sea otter inspired by Johannes Vermeer’s “Girl with a Pearl Earring” (1665), or teddy bears shopping for groceries in the style of Japanese Ukiyo-e prints.
OpenAI first introduced DALL-E, named with nods to the endearing robot protagonist of the 2008 Pixar movie WALL-E and Surrealist painter Salvador Dalí, in January of 2021 and has been working to refine the system ever since. DALL-E 2, the most current version, renders images in higher resolution based on greater comprehension of the prompts. It also has the added feature of “in-painting,” which enables a user to swap one aspect of a photograph for another — for example, seamlessly replacing a dog sitting on a chair for a cat, as demonstrated in an introductory video released by the company this month. Further, DALL-E can analyze an existing image and present an array of variations with different angles, styles, and colorways.
DALL-E leverages a two-stage model, first internally generating a “CLIP” image that matches with text based on deep-machine learning that has taught it to identify and correlate text with images, and then using a “decoder” that generates an image to meet the described conditions.
“We show that explicitly generating image representations improves image diversity with minimal loss in photorealism and caption similarity,” said an OpenAI research paper, published on the DALL-E 2 website. “Our decoders conditioned on image representations can also produce variations of an image that preserve both its semantics and style, while varying the non-essential details absent from the image representation.”
In non-clinical terms, if you want to see “a bowl of soup that looks like a monster, knitted out of wool,” well, now you can. “A palm with a tree growing on top of it” — why not? These and more are available on DALL-E’s Instagram, where you can decide for yourself if this is the next great art trend (though unfortunately you cannot purchase that Vermeer-esque sea otter as a poster) and DM them with ideas for image generation.
Like all of us, DALL-E is still learning, and has certain limitations. Some of these are flaws in the data pool — for example, mislabeled images that amount to teaching the AI the wrong word for something, which might then affect its output. Others are imposed restrictions on the software capabilities, which includes a content policy that bans hateful symbolism, harassment, violence, self-harm, X-rated content, shocking or illegal activity, deception, political propaganda or images of voting mechanisms, spam, and public health.
The software, for instance, did not completely understand the art historical implications of Hyperallergic’s request for “‘The Scream’ on a roller coaster,” or “a photo of a Jeff Koons balloon dog getting popped with a pin in outer space,” but the images are pretty satisfying nonetheless.
Currently, OpenAI is guarding their technology closely, generating images upon request but not allowing it for open use outside the company. They also will not generate images of real people, which means the photos of my tasteful beach wedding to Channing Tatum are on hold AGAIN.
This points to a pitfall of AI-generated imagery, and one that the company is seemingly preparing to address: The creation of realistic-looking false images presents a potential new buttress for fake news, a movement which has already led to geo-political destabilization and a global public health crisis in recent decades. It’s all fun and games when you’re generating “robot playing chess” in the style of Matisse, but dropping machine-generated imagery on a public that seems less capable than ever of distinguishing fact from fiction feels like a dangerous trend.
Additionally, DALL-E’s neural network can yield sexist and racist images, a recurring issue with AI technology. For instance, a reporter at Vice found that prompts including search terms like “CEO” exclusively generated images of White men in business attire. The company acknowledges that DALL-E “inherits various biases from its training data, and its outputs sometimes reinforce societal stereotypes.”
For their part, OpenAI is still controlling the technology and requiring that the use of their images include disclosure of their status as AI-generated, as well as the inclusion of a little color-bar logo in the lower right-hand corner of all images — but the ability to enforce such measures seems difficult to maintain if their product is eventually open for use at the scale of the entire internet.
For now, we are in that hopeful, playful part of tech development, where we wonder at the marvelous nature of our own invention. As the saying goes, the road to singularity is paved with “Otter with a Pearl Earring.”
This week, missed signs of previous life on Mars, the appeal of forged art, and why are blue whales singing in lower octaves?
All the Beauty and the Bloodshed forcefully posits multiple parallels between the world Nan Goldin grew up in and the one she fights in today.
The latest episode of this documentary series on PBS explores the meaning of home through handmade objects, hand built homes, and the artists who create them.
Your list of must-see, fun, insightful, and very Los Angeles art events this month, including Bob Thompson, Aimee Goguen, Uta Barth, the Transcendental Painting Group, and more.
There is the singular artist and then there is the more exclusive club that has only one member. Harvey belongs to the latter.
Rhode Island School of Design opens registration for its residential summer Pre-College program and year-round online intensive Advanced Program Online.
The artists say the Museum of Contemporary Art Kiasma must sever ties with Poju Zabludowicz, whose wealth comes in part from Israeli defense contracting.
Vanessa Albury, whose eco-friendly ceramic sculptures help revive filter-feeder populations, is raising funds to complete her first film about the project.
Hrag Vartanian, Hyperallergic’s editor-in-chief, is one of the guest jurors reviewing applications for the two-month residency in Utica, New York.
An archeological exploration of the amphitheater’s sewers and water systems uncovered remnants of meat, vegetables, olives, nuts, and yes, pizza.
At this year’s show, I reflected on the lack of bilingual materials, the absurdity of art-fair gimmick, and the workers who make it all possible.
Hear a band of improvisers led by Rajna Swaminathan and a performance of Morton Feldman’s “For John Cage” in programs inspired by the exhibition, “New York: 1962-1964.”
Your list of must-see, fun, insightful, and very New York art events this month, including art made during the first stock market crash, a homage to feline friends, and the 10-year anniversary of a crucial public art initiative.
Astrid Dick was told that she could not paint stripes because Sean Scully and Frank Stella have done so before her, a patently foolish statement.