Nfld. & Labrador

As software struggles persist, consultant says Muskrat Falls completion impossible to predict

A consulting company that's been monitoring progress on the Muskrat Falls project has released a stinging report that raises serious questions about when the controversial project might be ready for commercial operation.

Liberty Consulting says using customers as a 'test bed' for transmission software not normal

This is an aerial view of the sprawling switchyard at Soldiers Pond, outside St. John's, where DC power from Labrador will be converted to AC power and then integrated into the island power grid. (Nalcor Energy)

In one of its most scathing reviews to date, a consulting company monitoring progress on Muskrat Falls says it's now impossible to pinpoint a completion date for the troubled hydroelectric project.

In its latest quarterly report to Newfoundland and Labrador's Public Utilities Board, Liberty Consulting took aim at efforts to develop the power and control software for the Labrador-Island Link (LIL), which has suffered repeated setbacks and has been plagued by flaws.

The software is critical for the safe and efficient operation of the 1,100-kilometre line from Labrador to the Avalon Peninsula, which is capable of transmitting up to 900 megawatts of energy over two separate lines, or poles.

But the contractor, GE, has missed repeated deadlines for the software, and each new version exposes more and more glitches.

As such, the report's authors say "no reasonable projection of LIL commercial operation at full capability can have substantial credibility."

The report says it may take another year, "and perhaps significantly longer," to complete the project, and that N.L. Hydro must now prepare for "yet another coming winter season without a reliably performing LIL."

"Right now, the software will not support operation at any power level in a manner that system operators can consider dependable," Liberty reports.

Liberty has 'great concern' about methods

In its March 3 monthly update to the PUB, N.L. Hydro expressed similar concern, writing that trial operations are behind schedule.

"It is not possible to predict the extent of any possible delay at this point in time," reads the update.

Some 3,200 transmission towers like this one were erected as part of the $3.4-billion Labrador-Island Link from Muskrat Falls to Soldiers Pond. (Nalcor Energy)

It's yet another gloomy portrait of a project that is years behind schedule, billions over budget, and was the subject of a public inquiry that labelled Muskrat a misguided project and resulted in the dismantling of Crown-owned Nalcor Energy.

The project has ballooned to $13.1 billion, nearly $6 billion higher than the original 2012 estimate, and has required a multibillion-dollar bailout from the federal government to prevent electricity rates from skyrocketing in the Muskrat era.

Liberty Consulting is a U.S.-based firm that's been hired to monitor progress on the project, and its integration in the province's electrical system.

Following recent meetings with N.L. Hydro officials, Liberty said it had "great concern about the methods being employed to surmount very long-standing issues with LIL software."

Jennifer Williams is president and CEO of Newfoundland and Labrador Hydro. (Terry Roberts/CBC)

Liberty said it's taking an "extraordinarily long" time to perfect the software and that pressing trial versions of the software into service on the link, instead of using simulations, is not normal.

"Testing has continued to fail the purpose of finding and correcting defects before pressing the system and customers into service as the test bed for discovering remaining problems," Liberty wrote.

That the software here still suffers significant failings after so many years of development and failed testing may expose customers and the HVDC system equipment to risk. Good management should not consider doing so normal.- Liberty Consulting

"That the software here still suffers significant failings after so many years of development and failed testing may expose customers and the HVDC system equipment to risk. Good management should not consider doing so normal."

The link was energized in October, delivering up to 425 megawatts of energy, or roughly half the rated output of Muskrat Falls, for several months.

There was an unexpected shutdown, described as a bipole trip, in mid-December that affected about 22,000 customers. Liberty said such an incident should occur at a rate of no more than once every 10 years.

A more serious incident occurred Feb. 20 when a fire sensor caused one of the two lines to trip, and the entire link has been offline ever since because the software did not respond correctly. 

There was no fire, but Liberty said "that failure has produced yet another indefinite LIL shutdown while GE explores the additionally discovered and critical software deficiency, even as it continues to address other problems with the software."

GE will be 'held to account'

Hydro president and CEO Jennifer Williams, responding to the report Friday afternoon, said there's frustration over the ongoing delays. However, she said Hydro will not compromise on its software requirements, and that GE will be "held to account" on its contractual obligations.

 "I'm not as focused on the exact date. Our hard line is that it's got to be the right package of software," Williams said.

Williams also denied Hydro was taking unnecessary risks by operating the link with interim software. She said extensive offsite testing is done before the software is used on the link.

"We're not being cavalier. When we have an outage or a trip, it doesn't feel good. We're not using the system as a test bed, but it has to be placed in service at some point," she said.

Williams would not comment directly on Liberty's pessimistic outlook on the software, but stressed that a new version is expected in the coming weeks.

While it's possible the new link may not be fully functional by next winter, Williams said, customers shouldn't worry about the power grid's reliability. She said Hydro is committed to maintaining the Holyrood thermal generating plant for as long as it takes to ensure the system can meet the demand.

Meanwhile, she said an update on the project cost is expected this summer. While the $13.1-billion price tag is expected to grow, she said she doesn't expect it to be "significantly" higher.

Read more from CBC Newfoundland and Labrador

ABOUT THE AUTHOR

Terry Roberts is a reporter with CBC Newfoundland and Labrador, and is based in St. John’s. He previously worked for The Telegram, The Compass and The Northern Pen newspapers during a career that began in 1991. He can be reached by email at: Terry.Roberts@cbc.ca.

Comments

To encourage thoughtful and respectful conversations, first and last names will appear with each submission to CBC/Radio-Canada's online communities (except in children and youth-oriented communities). Pseudonyms will no longer be permitted.

By submitting a comment, you accept that CBC has the right to reproduce and publish that comment in whole or in part, in any manner CBC chooses. Please note that CBC does not endorse the opinions expressed in comments. Comments on this story are moderated according to our Submission Guidelines. Comments are welcome while open. We reserve the right to close comments at any time.

Become a CBC Member

Join the conversation  Create account

Already have an account?

now