Meet Rover, the AI app that's helping sniff out potholes in Markham

The City of Markham is collaborating with tech company Visual Defence to scour 1,100 kilometres of pavement for potholes, hoping that repairing them quickly will avoid costlier problems down the road.

Rover uses smartphone camera instead of expensive scanning equipment

Smartphone AI Rover on patrol for potholes in Markham. The pilot project will test if the system quickly and efficiently identifies roads in need of attention. (Chris Mulligan/CBC)

Potholes might look a a cheap and simple problem for cities to solve — after all, they're just holes in the road that need to be filled in with a bit of asphalt, right?

Well, actually, no.

Potholes are expensive to track down and fix, which is why the City of Markham is piloting a new smartphone-based artificial intelligence system that detects them more efficiently and at less cost — so they can be repaired before they become bigger, and more costly, problems down the road.

Called Rover, the app uses the phone's camera to spot damage that human eyes may miss, helping municipalities save money by making more timely road repairs. 

After a quick set-up, the phone is mounted on a cradle on a vehicle's windscreen, with the camera pointed at the road.

While city inspectors drive their routes checking on the city's infrastructure, such as signs, painted lines and sidewalks, Rover is keeping a close eye on the condition of the road. 

Alice Lam, the City of Markham's senior manager of roads, survey and utility, says initially two city vehicles will be equipped with Rover, helping staff detect problems faster and more efficiently.

"Having this device capturing the pothole location can save them time from stopping and having to mark down the location. It also eliminates human error."

Potholes are marked by a blue mark on the screen and the GPS coordinates are automatically logged so a road crew can be scheduled to return and fix them.  

Rover uses the smartphone camera to look for potholes and automatically log their locations using GPS. Road crews can return later to do the repairs. (Chris Mulligan/CBC)

"I have a separate crew to help us to do the pothole repair. That crew has a list of all the pothole locations. Boom, boom boom they can go out and fix all of them and one day," said Lam.

While there are other AI pothole detection systems, this one doesn't require thermal cameras, laser scanners and other sensors mounted to a vehicle, which makes it an affordable solution for municipalities. It's also easy to train employees to use.

But like a new puppy, the Rover artificial intelligence program must also be trained. 

Alice Lam, senior manager of roads for the City of Markham, says the system allows inspectors to concentrate on the state of repair of other infrastructure while the Rover AI keeps an eye on the roads for potholes. (Philip Lee-Shanok/CBC)

Roy Tal, the chief technology officer at Visual Defence Inc., based in Richmond Hill, Ont., says Rover relies on machine learning, so the program requires a lot of diverse data to be able to tell the difference between a pothole and a shadow, for example.

"You'd be surprised by how many different types of potholes there are," said Tal.

The company hopes this pilot will show that AI can easily and cheaply detect problems with roads, bridges and other infrastructure. 

Markham Mayor Frank Scarpitti says the city prides itself as a technology hub and says artificial intelligence can help city road crews cover more ground, effectively at less cost. (CBC)

It can be taught to spot cracks or water damage underneath the asphalt layer that will become problematic, or other warning signs, like water pooling on pavement.

"I would say that Rover is like a second set of eyes," said Tal.

"It gives the municipality's operator patrols that ability to have a more real time picture about what's going on on the road."

The City of Markham budgets $8 million annually to maintain 1,100 kilometres of road.

Mayor Frank Scarpitti says the city prides itself as a technology hub.

He says repairing potholes in a timely way not only saves a bigger expense later, but protects the city against legal claims by motorists for vehicle damage if they hit one.

"I think it's just cool technology. We're living in some pretty exciting times and I think it's really cool when technology allows us to cover more ground with less costs and in a more effective way."

As for any privacy concerns, the city says Rover is only collecting pothole images and locations. Any licence plates or anything captured that may identify an individual are blurred out.

But Scarpitti  says Rover won't replace calls from the public reporting nuisance potholes.

"Any member of the public can take a picture of anything they see that needs our attention, so we haven't eliminated what I call that neighbourhood watch type mentality."


Philip Lee-Shanok

Senior Reporter, CBC Toronto

From small town Ontario to Washington D.C., Philip has covered stories big and small. An award-winning reporter with more than two decades of experience in Ontario and Alberta, he's now a Senior Reporter for CBC Toronto on television, radio and online. He is also a National Reporter for The World This Weekend on Radio One. Follow him on Twitter @CBCPLS.


To encourage thoughtful and respectful conversations, first and last names will appear with each submission to CBC/Radio-Canada's online communities (except in children and youth-oriented communities). Pseudonyms will no longer be permitted.

By submitting a comment, you accept that CBC has the right to reproduce and publish that comment in whole or in part, in any manner CBC chooses. Please note that CBC does not endorse the opinions expressed in comments. Comments on this story are moderated according to our Submission Guidelines. Comments are welcome while open. We reserve the right to close comments at any time.

Become a CBC Member

Join the conversationCreate account

Already have an account?