New Project Pairs Modern News Photos with Old Masters
“Recognition,” winner of Tate’s IK Prize, uses machine learning to match artwork with images coming from the 24/7 news cycle
The Tate gallery in Britain holds some of the most iconic and interesting artwork from the last 500 years. But the gallery, like many arts institutions, wrestles with questions of how to make its collection interesting and relevant in the digital world. For the last three years, the Tate has experimented with ways to intermingle oil paints and pixels through its IK Prize for Digital Innovation.
This year’s winner, an Italian communications research center called Fabrica, created an artificial intelligence program called "Recognition." Over three months, beginning today, the autonomous program will scan through the Tate’s archive of 30,000 digital images of paintings and sculptures and match them with thematically or visually similar contemporary news images pulled from Reuters. Over that time, it will build a virtual gallery of images showing similarities between past and present.
“The team have created and trained a ‘brain’ to a point where it is simulating certain human attributes and unleashed it online – and it is creating a gallery,” Tony Guillan, the producer of the IK prize at the Tate Gallery tells Nicola Davis at The Guardian.
According to a press release, "Recognition" uses object and facial recognition as well as color and composition analysis. It will also analyze the captions that go along with the painting and photos to find similar subject matters. The project is using machine learning software developed by Jolibrain. “It can look for objects, like cups and saucers, it can look for faces, it can look for composition within an image – reading lines and colors – and it can look at the context that is attached to an image, so metadata, titles and things like that,” Isaac Vallentin of Fabrica tells Davis. “This process of going from specifics to the abstract concept, this is what is really challenging.”
One of the first “recognitions” the software has made is a comparison of the Dutch painter Peter Lely’s "Two Ladies of the Lake Family" (1660) with a news wire image of two eunuchs in Mumbai applying makeup before the Raksha Bandhan festival celebrations.
While the project will only last a few months, the Fabrica team sees a bigger canvas for their project. They write on their website:
“Imagine a smart machine that could learn to ‘read’ the ever-increasing digital archive that is the internet, learning to analyze millions of photographic images in terms of composition, colour, style and even content. What if this image-hungry ‘brain’ could also learn to ‘understand’ great artworks? Would it find interesting correspondences between our virtual present and our visual past? Do the ubiquitous images that fill our screens today bear any relation to the ways artists interpreted the world in art? Do they have similar aesthetic qualities; have similar subjects been represented in surprisingly similar ways?”
Fabrica will receive a $20,000 cash prize and $120,000 development budget. Other projects on the short list for the prize include: "The Wandering Intelligence of Art" which would give artwork virtual eyes and ears to look back at visitors and change based on its surroundings, "OSCAR," AI that would observe the artwork and visitors over a period of time before creating its own art and "Texting Tate," a texting chatbot that would learn how to describe the art in the gallery with the help of the public.