Q: My hospital deals with complicated surgeries that other hospitals don't have to deal with-- it's not fair to compare a hospital that does complicated surgeries to one that only does day surgery.
A: The CBC Rate My Hospital report card is based on empirical data reported by the hospitals themselves to the Canadian Institute for Health Information. CIHI has analysed the data and made adjustments to account for differences in hospital size and the types of cases they handle.
CIHI makes two types of adjustments to allow for fair comparisons between hospitals. A risk adjustment accounts for the fact that different hospitals see patients with different characteristics. It includes risk factors such as age, gender and whether patients have coexisting conditions that might influence how they fare in hospitals. CIHI also assigns hospitals to peer groups based on hospital size, a hospital's resources and the complexity of cases they see.
The CBC report card uses the same model. Each hospital's grade is calculated based on risk adjusted data published by CIHI, comparing performance between hospitals in the same size category as defined by CIHI.
Q: How can this "rating" be true if you don't have true data to validate claims? Acuity scores are not taken into account. My unit in my hospital gets complex cases from other hospitals. If that patient expires, that is a check mark on my unit, not the other hospital.
A: The mortality rates used in the Rate My Hospital report card were calculated and risk-adjusted by CIHI. According to CIHI's technical documents, these adjustments are made to "provide comparable indicators to support performance measurement and quality improvement among Canadian hospitals" and to "enable hospitals to compare themselves with other hospitals in their category."
When CIHI calculates mortality rates after major surgery, it assigns each death to the first hospital where a patient was treated for that episode of care. If a patient had major surgery in Hospital A, was immediately transferred to Hospital B, where he did not have major surgery but died within five days, the death is attributed to Hospital A.
Q: Assessing hospitals should be standard, but ranking them on a standardized scale is not good research. Take for instance the first indicator: death after surgery. After what type of surgeries exactly? Perhaps the hospital that received a D ranking performs more high-risk surgeries than other hospitals.
A: The indicator "deaths after major surgery" includes only surgeries that take longer than two hours, usually require a general anaesthetic and are done in an inpatient setting. It does not include day surgery. The types of procedures are defined by a coding manual developed by the World Health Organization, known as the International Classification of Diseases. They include procedures such as joint replacements, heart bypass surgery and colostomies.
Q: The report card could encourage Canadians to use "flawed information" to make decisions about where and when to receive hospital care without the advice of medical professionals.
A: CBC hospital ratings are not medical advice or recommendation. CBC does not advise Canadians to use them as their only source of information when making decisions about their medical care. The tool is intended as a starting point, to help patients ask their health care providers informed questions and to encourage hospitals to take a closer look at the care they provide.
Q: Hospitals are very large, very complex organizations. To rate anything on the basis of five measures is incomplete at best and potentially misleading at worst."
A: CBC decided on which measures to use in its ratings based on advice from an international panel of experts who specialize in assessing and improving the quality of hospital care. The five measures assess the quality of general surgical care and medical treatment only. Many of the indicators reported by CIHI focus on specialized care not offered by all hospitals, such as obstetrics and cardiac care. Including those measures would have significantly reduced the number of hospital CBC could have rated.
A low rating on these five measures does not mean a patient cannot get great care at that hospital. A high rating does not mean nothing will go wrong.
Q: There are no ratings anywhere in Quebec. What's up with that?
A: Quebec uses a different system from the rest of Canada to report hospital data. It is working with CIHI to harmonize its reporting system. But for the time being, only limited information is available about hospitals in Quebec.
Q: Letting people determine what grade their hospitals should have is unfair and irresponsible.
A: The patient rating tool included on the pages does not affect the overall grades awarded by CBC. It is there as a supplement, so that patients can compare their own experience to the results hospitals report themselves. Research in the U.K. has shown that online patient ratings such as these accurately reflect health care outcomes at hospitals.
Q: Basing grades on surveys sent to hospitals doesn't take into account relevant and complicated information about health care.
A: The fifth estate conducted three surveys during its investigation. Two of them - a survey of nurses and a survey of patients - were used exclusively to develop news stories about hospital care. There is no information from those surveys on the Rate My Hospital pages.
The third survey was sent to the CEOs of more than 600 Canadian hospitals. The CEOs of 139 hospitals replied. Their responses are reflected in "the fifth estate Hospital Survey" section of the report card but did not affect a hospital's overall grade.
Q: This information is too complicated for CBC to understand properly.
A: A panel of experts from Canada, the UK and the US helped us make sense of the data and translate it into information that is easily understood by us and the public. Here is a list of the expert panel members:
Bottle is a senior lecturer in medical statistics in the School of Public Health at Imperial College in London, U.K. He is also the chief analyst with the school's Dr. Foster Unit, which develops tools for displaying health care information and monitoring the quality of care. He devised the statistical methodology behind the national real-time monitoring system for tracking patient outcomes used by Dr. Foster Intelligence, the main company that analyzes health care information for the U.K. government's National Health Service. His research focuses on using routinely collected data to analyze variations in the quality of health services with an eye to helping hospitals improve patient care.
O'Connor is director of nursing and chief nursing officer at Montreal's McGill University Health Centre (MUHC) and an assistant professor in McGill's School of Nursing. She has been involved in numerous initiatives to improve patient care, including the MUHC's best practices program to reduce fall injuries and pressure ulcers and improve pain management. She is currently leading the Transforming Care at the Bedside program, which focuses on engaging patients and front-line staff in the redesign of systems that affect inpatient care. O'Connor has received numerous honours, including the Montreal Neurological Institute and Hospital's lifetime achievement award in 2009.
Rudolph is a senior scientist at the Center for Health Systems Research and Analysis at the University of Wisconsin-Madison and is the lead scientist for the County Health Rankings program at the university's Population Health Institute. She is interested in how health care data is collected and reported and how hospitals measure performance and quality of care. Rudolph helped develop several hospital ranking programs, including the U.S. News and World Report's Best Hospitals rankings, the Leapfrog Group's Hospital Safety Score, and the Wisconsin Bureau of Health Services' Consumer Guide to Health Care, which included hospital performance.
Teare is director of measurement and analysis for the Saskatchewan Health Quality Council, where he heads a team of researchers who collect and analyze information about patient care and safety from health care providers across the province to help them use it to improve their performance. Teare is also an adjunct professor in community health and epidemiology at the University of Saskatchewan in Saskatoon. He has worked with the Toronto Rehabilitation Institute and the Institute for Clinical Evaluative Sciences. He is also a member of the accreditation program advisory committee of Accreditation Canada, a non-profit organization that assesses health care institutions.
Tu is a cardiologist at the Schulich Heart Centre at Toronto's Sunnybrook Hospital and a senior scientist at the Institute for Clinical Evaluative Sciences, where he leads the cardiovascular and diagnostic imaging research team. He has published numerous studies evaluating the quality of cardiac care in Canada, including the landmark 2009 Enhanced Feedback for Effective Cardiac Treatment (EFFECT) study which examined how publicly reporting hospital performance data affects treatment. He was lead editor of the Canadian Cardiovascular Atlas, which mapped regional variations in cardiovascular care. Tu is a professor of medicine at the University of Toronto and holds the Canada Research Chair in health services research.
Q: I am a healthcare provider and was never told about the survey CBC sent out to staff at hospitals, why not?
A: CBC sent a survey to Registered Nurses across Canada through nursing associations and nursing unions in February. It's possible some nurses did not know about our survey. Were were not able to survey all hospital staff. We received responses from 4,500 nurses.
Q: My hospital is not on this list. Why not?
A: Our list is based on the list of hospitals that CIHI reports data for. If your hospital does not report to CIHI, the CBC was unable to rate it. We did not rate specialty facilities such as pediatric hospitals and urgent care centres.
Q: Will CBC release their data and exact methodology, including calculations for adjustments made?
The CBC's methodology is published online. You can see it here: http://www.cbc.ca/news/health/features/ratemyhospital/methodology.html