As It Happens

Someone cut the edges off this Rembrandt 300 years ago. Now AI has recreated the missing pieces

More than 300 years after someone sliced off the edges of a Rembrandt masterpiece, scientists have used an artificial intelligence to recreate the missing pieces. 

Rijksmuseum used artificial intelligence and an copy of the original painting to create the larger Night Watch

The Night Watch is put in place at the Rijksmuseum Museum on June 22, featuring an AI-generated version of the painting's long-lost missing pieces. (Remko de Waal/ANP/AFP/Getty Images)

Read Story Transcript

More than 300 years after someone sliced off the edges of a Rembrandt masterpiece, scientists have used artificial intelligence to recreate the missing pieces.

Now the public can see Rembrandt van Rijn's The Night Watch in its original size for the first time in centuries at the Rijksmuseum in Amsterdam. 

"As far as I know, this is the only time this has been done," Rijksmuseum scientist Robert Erdmann told As It Happens host Carol Off. "Also it's quite unusual to see the museum world collide with the artificial intelligence world."

This version of The Night Watch has been on display for hundreds of years, featuring Frans Banning Cocq and Willem van Ruytenburch at the centre of the action. (Fine Art Images/Heritage Images/Getty Images)

Rembrandt's 1642 painting was larger than the version that's long been on display at Rijksmuseum. That's because someone literally cut it down to size three centuries ago.

In 1715, the group portrait of an Amsterdam civil militia was moved from the militia's club house to the town hall. They planned to hang it on a wall between two doors, but it didn't fit. So they trimmed all four sides with scissors. 

The fate of the pieces of canvas that were sliced away remains a mystery. But luckily, art historians have an idea what the original looked like thanks to a smaller, 17th-century copy by Gerrit Lundens.

By lining up the copy with the original, and teaching and developing AI that can paint in the style of Rembrandt, the museum was able to recreate the missing pieces. The printed strips now hang flush to the edges of the original in the museum's Gallery of Honour.

It really gives the painting a different dynamic.- Taco Dibbits, Rijksmuseum director

In the chopped down version, the painting's two main characters, Capt. Frans Banninck Cocq and Lt. Willem van Ruytenburch, are centred. 

Now the duo is shifted to the right, as the artist intended, with the new digital additions of a group of onlookers to the left. The image of a drummer to the far right has also been slightly expanded. 

 "It can breathe now," said museum director Taco Dibbits. "It really gives the painting a different dynamic."

Viewers can compare the two versions in detail on the museum's website

Museum workers add a piece to Night Watch. (Remko de Waal/ANP/AFP/Getty Images)

Erdmann said lining up the virtual versions of the paintings was no easy feat, as they have key differences in scale, geometry, style and palette.

"This enabled the final step, which I like to call: sending the computer to art school," he said. 

He essentially cut up the two virtual paintings into thousands of tiles, then painstakingly taught the neural network to recreate each of Lundens' tiles in style of Rembrandt.

"And then we could grade the student because we have the actual corresponding title from The Night Watch," he said. 

"Once this training process is done, then we could ask this neural network to recreate the missing parts in the style, palette and scale of The Night Watch."

Robert Erdmann is the senior scientist at Amsterdam's Rijksmuseum. (Reinier Gerritsen/Rijksmuseum)

So why not just hire an actual person who's been to art school to do the work?

"That would have been a reasonable choice. But then it's quite likely that we would also get the style of that artist," Erdmann said. 

"That's fine, but we wanted to see if we could do that without having the hand of an artist in there — so the most direct translation from the style of Lundens to the style of Rembrandt.

Just don't call it a "restoration," Erdmann said. 

"This is really just a temporary exhibition in which we're sort of imagining what these pieces would be like, as contrasted to a restoration, in which we would actually physically modify the painting. We're not doing that in this case."

Still, as far as imaginings go, Erdmann says it's pretty good.

"Of course, there's no sense in which I would ever imagine that we could recreate the genius of Rembrandt," he said.

"I like to think that since Rembrandt was himself very innovative, he was willing to try new techniques constantly and invent new ways of doing things, that he would appreciate ... what we've done. The rest would be pure speculation."

Written by Sheena Goodyear with files from The Associated Press. Interview with Robert Erdmann produced by Niza Lyapa Nondo. 


Add some “good” to your morning and evening.

A variety of newsletters you'll love, delivered straight to you.

Sign up now


To encourage thoughtful and respectful conversations, first and last names will appear with each submission to CBC/Radio-Canada's online communities (except in children and youth-oriented communities). Pseudonyms will no longer be permitted.

By submitting a comment, you accept that CBC has the right to reproduce and publish that comment in whole or in part, in any manner CBC chooses. Please note that CBC does not endorse the opinions expressed in comments. Comments on this story are moderated according to our Submission Guidelines. Comments are welcome while open. We reserve the right to close comments at any time.

Become a CBC Member

Join the conversation  Create account

Already have an account?