Answers to frequently asked questions about Rate My Hospital

Here are the answers to some of the frequently asked questions about the fifth estate's special Rate My Hospital project.
CBC's Rate My Hospital project generated lots of questions and comments. (iStock)

Below are the answers to some of the frequently asked questions and comments the fifth estate received about its special Rate My Hospital project. Visit CBC's Rate My Hospital page for the results of its investigation.

My hospital deals with complicated surgeries that other hospitals don't have to deal with. It's not fair to compare a hospital that does complicated surgeries to one that only does day surgery.

CBC's Rate My Hospital report card is based on empirical data reported by the hospitals themselves to the Canadian Institute for Health Information (CIHI). CIHI has analysed the data and made adjustments to account for differences in hospital size and the types of cases they handle.

CIHI makes two types of adjustments to allow for fair comparisons between hospitals. A risk adjustment accounts for the fact that different hospitals see patients with different characteristics. It includes risk factors such as age, gender and whether patients have coexisting conditions that might influence how they fare in hospitals.

CIHI also assigns hospitals to peer groups based on hospital size, a hospital's resources and the complexity of cases they see.

The CBC report card uses the same model. Each hospital's grade is calculated based on risk adjusted data published by CIHI, comparing performance between hospitals in the same size category as defined by CIHI.

How can this "rating" be true if you don't have true data to validate claims? Acuity scores are not taken into account. My unit in my hospital gets complex cases from other hospitals. If that patient expires, that is a check mark on my unit, not the other hospital.

The mortality rates used in the Rate My Hospital report card were calculated and risk-adjusted by CIHI. According to CIHI's technical documents, these adjustments are made to "provide comparable indicators to support performance measurement and quality improvement among Canadian hospitals" and to "enable hospitals to compare themselves with other hospitals in their category."

When CIHI calculates mortality rates after major surgery, it assigns each death to the first hospital where a patient was treated for that episode of care. If a patient had major surgery in Hospital A, was immediately transferred to Hospital B, where he did not have major surgery but died within five days, the death is attributed to Hospital A.

Assessing hospitals should be standard, but ranking them on a standardized scale is not good research. Take for instance the first indicator: death after surgery. After what type of surgeries exactly? Perhaps the hospital that received a D ranking performs more high-risk surgeries than other hospitals.

The indicator "deaths after major surgery" includes only surgeries that take longer than two hours, usually require a general anaesthetic and are done in an in-patient setting. It does not include day surgery. Procedures include joint replacements, heart bypass surgery and colostomies.

The report card could encourage Canadians to use "flawed information" to make decisions about where and when to receive hospital care without the advice of medical professionals.

CBC hospital ratings are not medical advice or recommendations. CBC does not advise Canadians to use them as their only source of information when making decisions about their medical care. The tool is intended as a starting point, to help patients ask their health care providers informed questions and to encourage hospitals to take a closer look at the care they provide.

Hospitals are very large, complex organizations. To rate anything on the basis of five measures is incomplete at best and potentially misleading at worst.

CBC decided on which measures to use in its ratings based on advice from an international panel of experts who specialize in assessing and improving the quality of hospital care. The five measures only assess the quality of general surgical care and medical treatment. Many of the indicators reported by CIHI focus on specialized care not offered by all hospitals, such as obstetrics and cardiac care. Including those measures would have significantly reduced the number of hospital CBC could have rated.

A low rating on these five measures does not mean a patient cannot get great care at that hospital. A high rating does not mean nothing will go wrong.

There are no ratings anywhere in Quebec. What's up with that?

Quebec uses a different system from the rest of Canada to report hospital data. It is working with CIHI to harmonize its reporting system. But for the time being, only limited information is available about hospitals in Quebec.

Letting people determine what grade their hospitals should have is unfair and irresponsible.

The patient ratings tool included on the hospital profile pages does not affect the overall grades awarded by CBC. It is there as a supplement, so that patients can compare their own experience to the results hospitals report themselves. Research in the U.K. has shown that online patient ratings such as these accurately reflect health care outcomes at hospitals.

Basing grades on surveys sent to hospitals doesn't take into account relevant and complicated information about health care.

The fifth estate conducted three surveys during its investigation. Two of them — a survey of nurses and a survey of patients — were used exclusively to develop news stories about hospital care. There is no information from those surveys on the individual hospital profile pages.

The third survey was sent to the CEOs of more than 600 Canadian hospitals. The CEOs of 139 hospitals replied. Their responses are reflected in "the fifth estate Hospital Survey" section of the hospital profile pages but did not affect a hospital's overall grade.

This information is too complicated for CBC to understand properly.

A panel of experts from Canada, the United Kingdom and the the United States helped us make sense of the data and translate it into information that is easily understood by us and the public. For more on the five panel members, here are their bios.

I am a healthcare provider and was never told about the survey CBC sent out to staff at hospitals, why not?

CBC sent a survey to registered nurses across Canada through nursing associations and nursing unions in February. It's possible some nurses did not know about our survey. We were not able to survey all hospital staff. We received responses from about 4,500 nurses.

My hospital is not on your list. Why not?

Our list is based on the list of hospitals that CIHI reports data for. If your hospital does not report to CIHI, the CBC was unable to rate it. We did not rate specialty facilities such as pediatric hospitals and urgent care centres.

Will CBC release their data and exact methodology, including calculations for adjustments made?

CBC's methodology is published online.