Inspired by Russian painter Wassily Kandinsky, Google Arts & Culture launched the Play a Kandinsky experiment. The interactive project of synesthesia, a neurological condition that makes one sense trigger another, makes people hear “the sounds of colors”.
The artificial intelligence tool provides an experience similar to that of the expressionist artist. In this case, Kandinsky became known for creating paintings based on the colors and shapes he saw while listening to music.
Developed in collaboration with the Pompidou Center in Paris, the experiment imagines the melodies that the artist heard when painting the painting “Yellow, Red, Blue” from 1925. The work in question presents an impressive design with geometric lines.
Using the relationships between colors, shapes and sounds described by the artist, Google researchers trained the Transformer neural network with the songs of the time. From there, they used artificial intelligence to generate new scores.
“An extraordinarily talented artist, Kandinsky organized‘ colorful sounds ’into‘ color symphonies ’. Thus, the character of the colors reflected for him the harmonies and dissonances of a melody ”, describes the Play a Kandinsky team.
Collaboration of musical artists
Antoine Bertine, musician who collaborated with the project, reveals that the initial idea was to use artificial intelligence as an archaeological tool. Thus, he and the machine acted only as intermediaries between the work of the Russian artist and the current world.
Play a Kandinsky allows people to listen to an entire masterpiece as the artist may have heard while painting the picture. In addition, you can create your own compositions based on the original painting.
Despite not reproducing the connections between the images and sounds that a kinesthetic feels, the interactive project is an intriguing exploration of the connections between music and painting. Check out a video on the experiment creation process: