Experience Wassily Kandinsky’s Art Through Simulated Synesthesia

Synesthesia, present in 2-4% of the global population and poetically translated as a “joining of the senses,” is one of the most artistically darling mental conditions. It manifests differently in every synesthete, but generally connotes an unusual activation of multiple senses in response to stimuli. Typical examples are perceiving numbers as different hues, or conflating audio input with particular shapes and colors.

This was famously the case for painter Wassily Kandinsky, who was inspired by one of his colorful visions of a symphony to switch from a law career to study art. Kandinsky’s paintings are vibrant, abstract, and fluidly geometric compositions that seek to capture the symphonic experience of his senses. Now, a new project from Google Arts + Culture, “Play a Kandinsky,” invites users to learn about the condition and “play” his masterpiece, “Yellow-Red-Blue” (1925). The program uses machine learning and Kandinsky’s extensive color theories to create an interactive experience that interprets what the painter might have heard when working on or viewing this painting.

Aiding in this interpretation are musicians Antoine Bertin and NSDOS, who studied the painter’s writings about his multisensory perception. Google then applied machine learning to create a simulation tool that allows users to hover over aspects of the painting and hear what comes out of it. This experience is part of Sounds like Kandinsky, an extensive collaboration with the Centre Pompidou in Paris to create an immersive experience and remembrance of the artist and his work. Other virtual aspects of the exhibition include a tour of his 1940 oil painting “Sky Blue” and photos and descriptions of his Paris studio.

Not only is “Play a Kandinsky” a unique and personal look at an artist’s experience and process, but it’s also a fascinating embodiment of a condition that, like the art it inspired, can be somewhat abstract.