Too Rembrandt to Fail: Dutch Bank 3D Prints “New” Old Master

You may be quick to identify a portrait unveiled this week in Amsterdam as a never-before-seen painting by Rembrandt. With a calm gaze, mouth slightly parted, and wearing a frilled collar with a wide-brimmed hat, the man resembles the sitters the Dutch painter so frequently depicted. Rather than dabs of paint, however, this portrait consists of pixels — 148 million of them, to be exact, all created by machines and captured in a 3D-printed painting.

The artwork, dubbed “The Next Rembrandt,” is the result of 18 months of research and experimentation by a team of scientists, developers, engineers, and experts on the painter to use data collected from Rembrandt’s oeuvre to compose an entirely new painting. The final computer-generated work is strictly faithful to his hand and aesthetic, replicating the artist’s rendering of facial features, the colors and brushes he would have used, his typical geometric composition, and his careful shading and use of light.

“It’s not about mimicking Rembrandt,” project spokeswoman Jessica Hartley told Hyperallergic. “It’s more about trying to predict what he would do next on the basis of data. The painting is not made by Rembrandt, but it’s 100% Rembrandt because every decision that’s been made is based on data of his work.”

The project arose out of a request from ING Bank, as part of its sponsorship of Dutch art and culture, to conduct an experiment in innovation beyond the world of finance. The Amsterdam branch of advertising agency J. Walter Thompson took on the task, deciding it would teach a computer, armed with artificial intelligence, to learn from Rembrandt’s works to produce something new.

“We were seeing how far data and technology could take us,” the agency’s executive creator director Bas Korsten told Hyperallergic. “The outcome was highly unsure and unpredictable. We didn’t know if we would have a painting at the end, and what it would look like. One of the more surprising outcomes was how we, on a facial feature-by facial-feature basis, started to reconstruct a face. We could have ended up easily with a Mr. Potato Head.

“If I look at the painting now, I see a person with a soul, and he’s looking at me, and something’s happening behind the eyes. And I think that’s the most amazing thing that I’ve encountered.”

The team scanned the paintings they had access to and examined high-resolution images of other works held in private collections, creating a massive pool of data. Breaking down the demographics of Rembrandt’s subjects revealed that the painter mostly painted — unsurprisingly — white men between the ages of 30 and 40 who wore dark clothes with a white collar and were turned slightly to the viewer’s right. The researchers then designed a software system that incorporated facial recognition technology to identify the most common aspects of the artists’ paintings. Algorithms generated wholly new features and assembled them into a face and bust, all dictated by proportions in original Rembrandts, with systems calculating distances between features to churn out new, appropriate measurements that were rotated and scaled before being set in place. To add dimension, the team identified textures on the surface of all the canvases and with the help of engineers from Delft University of Technology, transformed the information into a “height map,” which determines the layers of paint in “The Next Rembrandt.” Based on this map, a 3D printer laid down paint-based UV ink on a surface to produce a Rembrandt realistic both in appearance and to the touch.

The team is currently looking at options for where its creation may go on public display. Art historians, according to Korsten, have already suggested the possibility of adopting this technology to restore paintings — using what he describes as “predictive modeling” to recreate lost fragments of works. Most of the algorithms used in the project will be available for public use, and he is eager to see how people may use the information. While some may bemoan machines’ increasing ability to replicate human touch and expression, Korsten is quick to reject the notion that such technologies are replacing our creativity.

“We taught a computer to look at existing Rembrandts, and it created a new work out of that,” Korsten said. “But we had to feed him with the creativity of Rembrandt. So as soon as the computer starts creating artwork by itself — and artwork that we actually like — then maybe it enters the territory of creativity, which I think is still unique to human beings. As soon as the computer starts innovating — and I think we’re not there yet by a long shot — then I think we can start worrying.”