USA. New report examines systems that rate the strength of scientific research studies and their findings

Citation

(2002), "USA. New report examines systems that rate the strength of scientific research studies and their findings", International Journal of Health Care Quality Assurance, Vol. 15 No. 5. https://doi.org/10.1108/ijhcqa.2002.06215eab.010

USA. New report examines systems that rate the strength of scientific research studies and their findings

USA

New report examines systems that rate the strength of scientific research studies and their findings

A new report, Systems to Rate the Strength of Scientific Evidence, sponsored by the Agency for Healthcare Research and Quality (AHRQ) identifies and compares systems that rate the quality of evidence in individual research studies and compilations of studies addressing a common scientific issue. The report also provides guidance on the leading approaches currently in use for improving the quality of scientific evidence.

In 1999, Congress mandated AHRQ to identify and disseminate "methods or systems to rate the strength of the scientific evidence underlying health care practice, recommendations in the research literature, and technology assessments". To address this charge from Congress, AHRQ commissioned its Evidence-based Practice Center (EPC) at RTI International-University of North Carolina to review these methods and systems.

In its review, the EPC identified 121 sources that deal with systems that rate the quality of individual studies-systematic reviews, randomised clinical trials, observational studies, and studies of diagnostic tests – or that grade the strength of bodies of evidence, including 12 reports from AHRQ-supported evidence-based practice centres. The EPC then evaluated the systems that rate the quality of individual studies using criteria based on findings from previous studies and best practices from clinical research for each of the four study designs. It also evaluated the systems for grading the bodies of evidence using three criteria – quality, quantity and consistency.

The authors summarised more than 100 sources of information on systems for assessing study quality and strength of evidence for systematic reviews and technology assessments. After applying evaluative criteria based on key domains to these systems, they identified 19 study quality and seven strength of evidence grading systems that those conducting systematic reviews and technology assessment can use as starting points. In making this information available to the Congress and then disseminating it more widely, AHRQ can meet the congressional expectations set forth in the Healthcare Research and Quality Act of 1999 and outlined at the outset of the report. The broader agenda to be met is for those producing systematic reviews and technology assessments to apply these rating and grading schemes in ways that can be made transparent for groups developing clinical practice guidelines and other health-related policy advice. The authors have also offered a rich agenda for future research in this area, noting that the Congress can enable pursuit of this body of research through AHRQ and its EPC programme. They are confident that the work and recommendations contained in this report will move the evidence-based practice field ahead in ways that will bring benefit to the entire health care system and the people it serves.

AHRQ Acting Director Carolyn Clancy, M.D. said: "this report will be invaluable to researchers and policymakers in evaluating the quality and strength of scientific evidence in an ever-growing sea of research. Researchers also will be able to use this information in future research design, to better ensure the quality of their work."

The full report, Systems to Rate the Strength of Scientific Evidence, is available by calling the AHRQ Publications Clearinghouse at 1-800-358-9295 or sending an E-mail to ahrqpubs@ahrq.gov and is available in a downloadable zipped file from http://www.ahcpr. gov/clinic/strength. The summary is available online at http://www.ahcpr. gov/clinic/epcsums/strengthsum.htm or through the AHRQ Publications Clearinghouse. Later in the year, the full report will be available at http://www.ahrq.gov/clinic/epcix.htm

Joint Commission on Accreditation of Healthcare Organizations approved as "deeming" authority

The Balanced Budget Act of 1997 directed the Centers for Medicare and Medicaid Services (CMS) to establish and oversee a programme that allows private, national accreditation organisations to "deem" that a Medicare€+€Choice organisation meets certain Medicare requirements. In March this year CMS announced the approval of the Joint Commission on Accreditation of Healthcare Organizations (JCAHO) as a deeming authority of Medicare€+€Choice organisations that are licensed as health maintenance organisations (HMOs) and preferred provider organisations (PPOs), making it easier for health plans to serve Medicare beneficiaries.

CMS found that the Joint Commission on Accreditation of Healthcare Organizations (JCAHO) standards for Medicare managed care organisations meet or exceed those established by the Medicare programme. Tom Scully, CMS administrator said that deeming helps to ensure that the Medicare€+ Choice plans are continuing to provide high quality service to their beneficiaries. He said that JCAHO had earned the right to act in that capacity and that CMS expects other organisations to do the same in the near future.

Medicare+Choice organisations that are licensed as HMOs and PPOs and are accredited by JCAHO may receive, at their request, deemed status for the Medicare€+€Choice requirements in six areas: quality assurance, information on advance directives, antidiscrimination, access to services, provider participation rules, and confidentiality and accuracy of enrollee records.

Improving the quality of laboratories doing simple tests

The Centers for Medicare and Medicaid services (CMS) has begun a new initiative designed to help physicians and other providers that perform simple laboratory tests provide better care to their patients. Laboratories performing these simple tests, known as "waived tests", are exempt from many of the federal regulations that apply to labs that do more complex testing. Waived tests are commonly performed in physician office laboratories, skilled nursing facilities, rural health clinics, and pharmacies, where on-site testing speeds the delivery of appropriate treatment.

CMS will be sending specially trained surveyors to a small, random sample of waived labs each year, using an approach that proved successful in a recently completed pilot programme. The laboratories will be notified in advance of the visits and encouraged to participate. Surveyors who find problems will work closely with lab personnel to help them better understand proper procedures and the implications for patient care of improperly performed tests. There are no fines or other sanctions associated with these surveys.

Tom Scully, CMS administrator said: "the goal of these surveys is to help physicians and other health care providers care for their patients with the aid of reliable test results without fear of new regulations or federal sanctions.

"In pilot studies, CMS has found that most providers welcome the educational approach and that testing practices improve in response to the program. This is consistent with our overall effort to work cooperatively with providers to improve quality."

All test systems that are waived under the Clinical Laboratory Improvement Amendments of 1988 (CLIA) must be accurate and simple to perform. In recent years, the number of waived test devices has expanded greatly, as has the number of laboratories performing these tests. Waived laboratories now constitute 55 per cent of the 175,000 laboratories performing testing in the United States and are exempt from standards in the CLIA rules for laboratory personnel, quality assurance and quality control. They are required to have a certificate of waiver from CMS.

Recent reviews by CMS and others have found problems with the quality of some of the tests performed. For example, test strips for a testing device may be placed on the device upside down. Kits may not be stored properly, or may be used after the expiration date. These errors create the potential for inaccurate results that may have an adverse impact on patient care. CMS expects the new educational effort will improve the quality of simple testing performed by these laboratories.

IWA-1: 2001 – Quality Management Systems: Guidelines for Process Improvements in Health Service Organizations

Quality Management Systems – Guidelines for Process Improvements in Health Service Organizations is based on ISO 9004:2000 which is more applicable to service organisations, such as health care, than the previous ISO 9000:1994 series. The guidelines were developed by the Automotive Industry Action Group (AIAG) in conjunction with the American Society for Quality (ASQ) Healthcare Division and subsequently released by the International Organisation for Standardisation (ISO). The guidelines can aid in the development or improvement of a fundamental quality management system within health care organisations. This system provides for continuous improvement, enhanced overall patient care, more effective and efficient services and reduced health care costs.

R. Dan Reid, member of AIAG's Healthcare Project Team and manager of advance activities for worldwide purchasing at General Motors Powertrain said: "ISO 9000 has driven significant improvements in organizations throughout the world, with more than 400,000 registrations issued to date. An ISO 9000-based system was used in the automotive industry with great success, so it seemed logical to take that knowledge and apply it to healthcare."

Several provisions have been included in the document to benefit organisations and their stakeholders. These guidelines provide the quality system basics, some of which are not addressed in some current health care accreditation criteria. The AIAG and ASQ recommend that health care organisations use the Quality Management Systems – Guidelines for Process Improvements in Health Service Organizations document to help define the basic processes and procedures of their quality system. It is intended to be used in conjunction with relevant health care accreditation criteria and any other applicable guidelines, although the guidelines are voluntary, and not intended for accreditation or certification.

The guidelines can be purchased from AIAG's Customer Service Department at (248) 358-3003 or through the ASQ Web site at http://qualitypress.asq.org/perl/catalog.cgi? item=T300

Initiative for quality improvements

In April, the Robert Wood Johnson Foundation (RWJF) and the Institute for Healthcare Improvement (IHI) announced that 12 health care organisations will continue to seek dramatic improvements in the care they provide patients in the second phase of the $20.9 million initiative, Pursuing Perfection: Raising the Bar for Health Care Performance.

Risa Lavizzo-Mourey, MD, senior vice president and director of the Health Care Group of the Robert Wood Johnson Foundation. said: "we think that all the organizations participating in Pursuing Perfection can help the nation understand that near-perfect health care, as shown by measurable results, is within our grasp. We expect that these organizations will produce compelling examples that show health care professionals and the public just how good our health care can and should be."

The Robert Wood Johnson Foundation launched the Pursuing Perfection programme in May 2001 with the release of survey findings that showed more than half of health care providers and administrators believe the overall quality of health care in the USA is not good. The survey also indicated that 80 per cent of health care providers believe fundamental changes are necessary in the health care system. The framework for Pursuing Perfection is outlined in the Institute of Medicine's report, Crossing the Quality Chasm. The report outlines six aims for improving health care: safety, effectiveness, patient-centredness, timeliness, efficiency and equity.

The 12 organisations participated in a seven-month initial phase under which they received funds to develop comprehensive plans for systematically improving health care quality. In the second phase, seven of them will receive two-year grants of $1.9 million each to help implement their plans. The others will continue to participate in the programme implementing all or part of their plan.

Donald M. Berwick, MD, MPP, president and CEO of the Institute for Healthcare Improvement, said the goal of the initiative was to make it possible for patients to be measurably safer, healthier, functioning at higher levels, waiting less time for care, and feeling more respected by the health care system. He said that all 12 organisations had demonstrated a deep commitment and the will to contribute to the never-ending pursuit of perfection, and added: "we are confident that all of these outstanding organizations will continue their efforts to pursue perfect care and will become models of care that the world can emulate."

New national QIO effort: improving nursing home quality of care

A new federal initiative spearheaded by quality improvement organisations (QIO) will help nursing homes improve care for residents who often suffer from pain, delirium, depression, pressure ulcers, and loss of everyday functions. In 2002, working under contract to the US Department of Health and Human Services (HHS), QIOs will begin providing nursing homes with materials and technical support needed to upgrade clinical and organisational systems.

The QIO initiative complements a move by the Centers for Medicare and Medicaid Services (CMS) at HHS to publicly report on the quality of care at every Medicare and Medicaid participating nursing home. CMS will focus on indicators of quality selected by the National Quality Forum such as pain, delirium, re-hospitalisation, walking, pressure ulcers, psychotropic drug use, assisted daily living decline, weight loss, infections, and restraints. QIO teams will help the public understand and use the indicators in selecting nursing home facilities.

QIOs in nearly every state have worked with nursing homes on specific projects to improve care. QIOs will draw on this experience, as well as partnerships forged with state agencies, health plans, professional and industry associations, and consumer advocacy organisations to broadly improve nursing home quality of care. Current projects include assistance in the prevention and treatment of pressure sores, falls prevention, pain management, development of quality measures for rehabilitation services, improving diabetes outcomes, improving anticoagulant use, and immunisation campaigns.

QIOs will help nursing home management identify what is necessary to create a quality improvement culture and empower staff to build quality improvement processes into everyday work. They will give all nursing homes materials providing guidelines for proper care, methods for improving care, staff training information, model policies and protocols, and tools for assessing care, and will facilitate regional nursing home alliances to help facilities learn from each other and train staff to implement shared lessons and best practices.

In addition, QIOs will offer intensive technical assistance to a significant number of nursing homes in each state. In facilities that volunteer to participate, QIOs will help staff identify leadership roles, establish clinical care teams, and learn a process for continuously improving quality of care. Focusing on specific clinical indicators, teams will perform clinical assessments, establish new policies and treatment protocols, provide additional staff training, and assess whether the changes cause sustainable improvement in care.