Social-tagging results in a museum visit (photo by the author for Hyperallergic)

Established art museums aren’t really known for responsiveness to the individual wants of visitors. Theoretically, the best art exhibitions respond to the emotional and conceptual requirements of their time, rather than to individual tastes. In practice, it is beholden to the Patron (the elite group of funders who underwrite museums) and not the patron (your everyday ticket-holding museum-goer). To understand how and why museums make decisions, it’s often most useful, as the evergreen adage goes, to follow the money. Instagram and Tumblr have filled in some of these gaps, while the rise of a “Spotify for art lovers” streams visual art by theme. Now, an initiative from SFMOMA has art lovers texting instead of scrolling. Since June, a text message to the museum phrased “Send me _____” returns a work from its permanent collection related to nearly any word — or even emoji. “Send Me SFMOMA” has received 4.3 million text messages.

This initiative may seem like a one-off bid for visitor attention, but it actually comes out of a decade of art museum technology and research. If art museums were machines, we’d say their user interface has improved greatly over the last ten years. As it turns out, we have Steve to thank — and not that Steve.

An interaction with “Send me SFMOMA” (photo by the author for Hyperallergic)

In 2004, an information architect named Thomas Vander Wal coined the term “folksonomy” to describe language that originates from the masses up. He and other information architects were observing this process on the internet through social tagging — users generating and pairing keywords to images to make them easier to find later. A folksonomy behaves like a taxonomy in that it is meant to assign meaning to things by classifying them. But while a taxonomy strives for scientific accuracy, a folksonomy is by nature imperfect. The internet has democratized cultural sectors whose vocabulary was defined by a small elite; the art museum community, inspired by Vander Wal, is one of them.

Susan Chun was on staff at the Metropolitan Museum of Art in New York City when she read about Vander Wal’s work. At the time, the museum was aware that many people visiting its digital archives were struggling to find what they were looking for. It was a classic case of jargon vs. conventional language, Chun said. If the Met could just find a way to get the public to catalogue the archives themselves, then they would know that the system was user-friendly.

“Honestly, until tagging came along museums had so little access to how visitors understood works of art,” Chun says. Academically inclined by nature and training, staffers were insulated from the “glory, confusion, [and] distaste of the visitor.” In 2005, Chun banded with other like-minded museum professionals to found Steve: The Museum Social Tagging Project. Funded by grants from the U.S. Institute of Museum and Library Services, they eventually developed an open-source software to make tagging easy for any museums who wished to participate.

Take this painting, for example:

Winslow Homer, “The Gulf Stream” (1899), Oil on canvas 28 1/8 x 49 1/8 inches (photo by David Torcivia via Flickr)

Winslow Homer painted “The Gulf Stream” in 1899; it hangs in the Met. The team conducting The Steve Project was surprised when multiple people tagged the shark-laden painting with the word “dolphins.” Chun was fascinated by the “imperfect image” — the passing impression that a visitor gets of a painting. “The real goal is to bring people to works of art, however misremembered,” she says. Vander Wal explains that tagging is a means of retrieval. The way we store things to come back to them later is subjective. Often these associations are shared by others — hence why social tagging works. But museum professionals eventually arrived at the conclusion there is no point in using people to tag works for elements that computers can detect, like color. It is better to have people describe things computers can’t detect, like how does the piece make them feel?

By 2014, when I first visited the topic of social tagging, it was a buzzword out of vogue, and trying to motivate people to tag was just plain difficult, unless they were already art enthusiasts. The Brooklyn Museum turned tagging into a game called “Tag! You’re It” and another game called “Freeze Tag” to remove inaccurate or  inappropriate tags. Both games were discontinued due to low participation rates.

Museum studies scholar Ross Parry argues that the idea of a technological museum isn’t newsworthy. He says we are in a “postdigital” museum environment, in which museums will be judged not based on whether they have an app but on how well “digital thinking” is applied to all levels of the organization.We now visit museums to see works that are already available online. What happens when virtual reality can recreate the artwork in our home?

Sebastian Chan, formerly director of Digital & Emerging Technologies at Cooper Hewitt,  now works at the Australian Centre for the Moving Image (ACMI). It’s a question ACMI has already grappled with, as all of the films in their collection can be viewed in a visitor’s living room. Currently, the most important job of the museum is to provide critical context. While machines tag films frame by frame, the curators interrogate bias and find references between directors that a machine would miss. We’re “tagging the museum’s stuff as well as tagging the world,” says Chan. “The museum is now much bigger than what you see when you walk in the door,” because of the wealth of information that lives online. He compares it to the TARDIS, from Doctor Who: police telephone box on the outside, expansive spacecraft on the inside. The postdigital museum allows visitors to extract more data from the collection (and the collection to extract data from the visitors.)

The Steve project ignited a “radical shift toward how visitors saw things,” Chan says. Chan was part of the team that launched the Pen at Cooper Hewitt, the Smithsonian Design Museum in New York City. Now, every visitor to the converted Upper East Side mansion is offered a rubber-tipped stylus the length of a toothbrush. By pressing the Pen to object labels, visitors can save the object to their personal collection, available online via the code on their museum ticket. (In the first 75 days of the Pen, it was used by 93% of visitors.) Cooper Hewitt offers info about the objects one engaged with — their colors, country of origin, and yes, their tags.

Folksonomy is far from a dead language. Social tagging proved a critical stepping stone for museums striving for Parry’s postdigital state. What was losing relevance in 2014, has shown by 2017 to be the germ of art museum democratization — whether through physical interventions like the Pen, or new iterations of the tagging system.

“We had experimented with Steve as a product suite and an idea, but we found that the ‘folksonomy’ didn’t provide the sort of connectivity and richness that you get from having a few dedicated staff [tagging],” says Keir Winesmith, Head of Web and Digital platforms at SFMOMA. The museum has been tagging its digital collection for internal purposes for nearly a decade, but a project as personal as the one just launched “wasn’t on the radar,” says Winesmith. The museum can only display 5% of its collection at a time, so “Send Me SFMOMA” was really more about “bringing to light the work in the collection that was largely hidden” than crowdsourcing a list of terms. However, Winesmith says they couldn’t have done it without the tags: “it would have been a non-starter.”

Of course, the museum is learning from the gaps between user desire and deliverability — and not just NSFW asks trying to trip the program up. 75% of the texts get a match. Subjective words, like beauty, have been hard to match —  as were emojis, initially. “We ran this pilot for four days in May and the first thing we learned was really we’re not very good at emojis and most of them didn’t match anything,” says Winesmith. “We actually gathered together a whole bunch of people that know more about emojis than I do and we kind of had an emoji workshop and they explained what all the hidden meanings were.” The museum now has 450 emojis in their system. About two thirds of users are sending words, and a third are using emojis.

Susan Chun can’t get away from tagging. Timed with the 25th anniversary of the Americans with Disabilities Act, she implemented a program called Coyote at the Museum of Contemporary Art Chicago where she is currently Chief Content Officer. The museum staff committed to creating rich descriptions of every image on their website, as if they were describing them to a visually impaired friend. Chun was delighted to discover that the museum had some closet poets, but even moreso that visually impaired visitors weren’t the only ones to benefit. She calls Coyote a “powerful tool for people of all sorts who feel a little alienated in an art museum.” Language as emotional engagement and catharsis. As with Steve, Coyote is an open source tool.

The Brooklyn Museum, meanwhile, has shifted from the needs of the digital to the physical visitor. Sara Devine, the Manager of Audience Engagement & Interpretive Materials, is clear about this distinction. The biggest desire of museum visitors was to ask staff questions about what they were seeing, says Devine, so the museum launched the Ask app to allow them to do just that. There is no take-home component of the app, and users are anonymous. But while visitors can’t log in from home and continue to engage with the collection, their questions become a part of the museum’s metadata that informs how they organize the exhibits.

Economic restraints push museums to continuously re-evaluate their purpose, says Chan. “The business model of the museum has to include the public as well; technology is a means to deliver public value.” He echoes a thought he expressed when evaluating the initial success of Cooper Hewitt’s pen. “The Pen can only be seen as successful in so far as it enables the museum and its visitors to do more with our collection.” Even a physical intervention into the visitor experience ends with metadata about word association.

“There’s oftentimes language that is a barrier to people’s engagement with work,” says Winesmith. “That’s why [Send Me] is so ‘dumb.’. We didn’t want an expert way of using it, there’s just one way of using it.” The museum is in talks with half a dozen other museums around the world to adapt the program to their collections, including The Tate and the Auckland Art Gallery.

Writing for The Atlantic in 2015, amid buzz about the internet of things, Robinson Meyer deftly characterized Cooper Hewitt’s collection as things of the internet. The key to interpreting these objects or artworks lies not in the five senses, but in language. In this way, the pen is not mightier than the word. They need each other.

Daisy Alioto is a culture writer living in Brooklyn. Her work has appeared in Artsy, Modern Magazine, Curbed, Wallpaper* and more.