Can an Algorithm Determine Art History’s Most Creative Paintings?

“Creativity scores for 1710 paintings from Artchive dataset. Each point represents a painting. The horizontal axis is the year the painting was created and the vertical axis is the creativity score.” (screenshot by the author from arxiv.org) (click to enlarge)

Two computer scientists have developed an algorithm that measures creativity in visual art. At the end of the month Ahmed Elgammal and Babak Saleh, an associate professor and PhD candidate, respectively, in the Department of Computer Science at Rutgers University, will present their paper “Quantifying Creativity in Art Networks” at the International Conference on Computational Creativity, chronicling the results of their attempt to create a computer algorithm that tabulates the originality and influence of paintings. The results are mixed; Johannes Vermeer, Claude Monet, and Georgia O’Keeffe fare well, while Jean-Auguste-Dominique Ingres, Albrecht Dürer, and Camille Pissarro compute as comparatively uncreative. A closer look at Elgammal and Saleh’s methodology offers clues as to why.

Hans Memling’s “Martyrdom of St Ursula” (ca. 1489) and Francisco de Goya’s “Crucified Christ” (1780), artworks deemed uncreative and creative, respectively, by the creativity algorithm. (illustration by the author for Hyperallergic) (click to enlarge)

Their algorithm was developed by using computer vision technology to assess the visual characteristics of the 1,710 artworks in the Artchive database and 62,254 works on WikiArt. They focused specifically on paintings, and even more specifically on “Western paintings,” while “removing genres and mediums that are not suitable for the analysis such as sculpture, graffiti, mosaic, installation, performance, photos, etc.” Analyzing the images from the two databases, the algorithm isolated a certain number of visual concepts (as many as 2,659) in each artwork based on the presence of depicted objects like crosses, horses, faces, and so on. The algorithm, in other words, was calibrated for representational art, which explains the off-the-charts creativity readings generated by abstract art from the early 20th century.

More telling, however, is the definition of “creativity” the pair used to guide their research. “Among the various definitions of creativity it seems that there is a convergence to two main conditions for a product to be called ‘creative,’” they wrote. “That product must be novel, compared to prior work, and also has to be of value or influential.” Consequently, the algorithm promoted works that boasted unprecedented visual features as evidence of their novelty, and championed those it deemed “influential” if those original features reappeared in later works. For instance, a Dürer portrait from 1514 scored very low on the algorithm’s creativity scale, while Kazimir Malevich’s “Red Square” (1915) was found to be one of the most creative artworks of the last 600 years. Little allowance is made for artists working in established genres or with frequently recurring subjects like, for instance, most religious paintings. The algorithm champions newness, not sustained excellence in a specific genre.

“Creativity scores for 62K painting from the Wikiart dataset. The horizontal axis is the year the painting was created and the vertical axis is the scaled creativity score.” (screenshot by the author from arxiv.org) (click to enlarge)

What shortcomings Elgammal and Saleh’s research may have in the case of specific works or artists’ oeuvres it may make up for in its analysis of broader art historical trends. “The general trend in [the graph above] shows peaks in creativity around late 15th to early 16th century (the time of High Renaissance), the late 19th and early 20th centuries, and a significant increase in the second half of the 20th century.” In other words, according to the algorithm, the last 50 years have been a time of intense artistic creativity on a par with the Renaissance. The algorithm also determined that the year 1700 marked the nadir of creativity in Western art history.

The researchers, perhaps anticipating an Obama-caliber backlash to their apparent attempt to automate art history connoisseurship, make a point of including a disclaimer. “Our goal is not to replace art historians’ or artists’ role in judging creativity of art products,” they wrote. “In most cases the results of the algorithm are pieces of art that art historians indeed highlight as innovative and influential. The algorithm achieved this assessment by visual analysis of paintings and considering their dates only.” Impressive, but a true test of the creativity algorithm would be to task it with evaluating the novelty and influence of Zombie Formalism.

h/t MIT Technology Review