Machine learning and the art of forgery detection

Posted: January 16, 2025

In 1937, Abraham Bredius, an eminent art historian, declared Christ and the Disciples at Emmaus to be "the masterpiece of Johannes Vermeer.” He said, “it is a wonderful moment in the life of a lover of art when he finds himself suddenly confronted with a hitherto unknown painting by a great master, untouched, on the original canvas, and without any restoration—just as it left the painter's studio…”.^[1] Unfortunately for Bredius, the painter’s studio that the piece had just left was not, in fact, Vermeer’s.

Five years earlier, Dutch painter Han van Meegeren had rented a house in a tiny French village. He bought 300-year-old canvases, created his own badger-hair paintbrushes and began mixing his own pigments in the same way the classical masters did—white paint from lead and blues from lapis lazuli. But van Meegeren had a more sinister motivation than simply emulating the artists. He wanted to prove himself by creating a perfect replica 17^th-century painting that would fool the art world.

To make the paints he was using appear older than they were, he mixed them with Bakelite, then baked the paintings to harden them, before washing them with black ink to fill in the cracks. Several of these such paintings were in the style of Vermeer, including one that he named Christ and the Disciples at Emmaus.

Surprisingly, no one discovered how wrong Bredius was until van Meegeren confessed to the forgery himself in an unlikely defense against collaborating with the Nazis. Faced with one of van Meegeren’s paintings today, it’s likely that tech would have exposed them for the forgeries they were long before the artist felt compelled to confess.

The current state of the art

These days, museums and conservation studios scan artworks using macro x-ray fluorescence (XRF), a non-invasive imaging technique that helps identify chemical elements. When experts bombard a painting with X-rays, the different chemical elements in the paint each emit a unique, identifiable pattern of radiation.^[2] These patterns of fluorescent radiation let experts map the different chemical elements used on all layers of a painting, including those not visible to the naked eye.^[3]

With these insights, experts can identify the techniques, pigments and processes an artist used, as well as establishing whether previous conservation work has been carried out on a painting. XRF can also be used to confirm authenticity, detecting art forgery by highlighting anachronistic pigments.^[4]

But macro x-ray fluorescence generates huge amounts of complex data in the form of millions of XRF spectra, which tell us about the specific elemental composition of areas of a painting.^[5] Manual analysis of this volume of data isn’t feasible. Most current analyses need specific scientific expertise, and often rely on assumptions and estimates about which chemical elements were most likely to have caused the pattern of XRF spectra in the data.

The art of training an algorithm

In September of last year, a team of chemists and researchers at the Istituto di Scienze del Patrimonio Culturale (ISCP), Italy published a paper about their trial that used machine learning to analyze this large volume of data.^[6]

The ISCP team have worked on a deep learning algorithm that allows for fast and accurate analysis of the XRF spectra in macro x-ray fluorescence datasets. They trained their deep-learning model on a dataset of half a million synthetic spectra, which represented 57 pigments and compounds. The results showed that the model detected not only the correct chemical elements used in paintings but also the amount of each chemical element in the XRF spectra.

Testing machine-learning on Raphael

The researchers then tested their methodology on two paintings by Raphael, both on display at the Museo di Capodimonte in Naples. The wood-panel paintings, God the Father and Virgin Mary, represent two of only four preserved fragments of the grand Baronci altarpiece Raphael painted at the start of the 16^th century.

Researchers found that the pigment palette identified by the algorithm aligns with 15th-century practices and matches palettes known to be used by Raphael in his early work. The machine learning correctly identified the chemicals used in the painting. It deduced that lead was used in the white preparatory area and highlights, red vermillion in skin tones, copper green in draperies, and iron and manganese oxides in the two figures. The scans also detected that restorative work had occurred, using pigments from other eras.

To further demonstrate the accuracy of their approach, the researchers compared their results to analysis of the painting using traditional macro x-ray fluorescence analysis algorithms. They found that the machine-learning approach provided superior results, as the traditional method sometimes generates false positives or overestimates of a chemical’s presence. As well as better results, they found that the machine-learning model substantially reduced how long it took to analyze the results and required less expert involvement.

Preserving art for future generations

This ability to more accurately identify the distribution of different chemical elements used in paints improves our understanding of not only the materials an artist used, but gives insight into an artist’s techniques, shedding light on our heritage. Importantly, more accurate results mean that specialists can conserve and restore paintings more sympathetically, helping us preserve historic art for generations to come. The additional information about techniques such as brush strokes can help authenticate provenance alongside chemical testing.

Of course, the quality of art forgery often keeps pace with improvements in technology. Some forgers are starting to learn from the techniques applied by conservationists to detect fakes, for example using their own handheld XRF guns to check their work. In 2015, following a tip, French police seized a 16^th-century painting of Venus displayed in a gallery in Provence. Art historian Bendor Grosvenor said that the painting may turn out to be one of “the best old master fakes the world has ever seen.”^[7] Analysis of the paints used in the painting went on to throw doubt on the provenance of several artworks sold by the same collector.^[8] The battle between forgers and science is far from over, but with XRF progress, perhaps science has a brief upper hand.

^{[1] https://www.essentialvermeer.com/misc/van_meegeren.html
[2] https://www.bbc.co.uk/programmes/articles/5BYtkcLZ0WPcB1v45pFwL36/x-ray-fluorescence-xrf
[3] https://www.science.org/doi/10.1126/sciadv.adp6234
[4] https://www.sciencedirect.com/science/article/abs/pii/S0026265X17307051
[5] https://www.sciencedirect.com/topics/materials-science/x-ray-fluorescence-spectroscopy
[6] https://phys.org/news/2024-10-ai-figure-chemical-composition-classical.html
[7] https://www.theguardian.com/news/2018/jun/15/how-to-spot-a-perfect-fake-the-worlds-top-art-forgery-detective
[8] https://www.theartnewspaper.com/2016/11/01/old-masters-sold-by-giuliano-ruffini-face-testing-times}