Outcomes and Effectiveness

 

Joseph H. Kanter Family Foundation and eHealth Initiative Foundation Launch Partnership to Conduct Research on Health Outcomes, Comparative Effectiveness

Multi-stakeholder effort will pilot a unique, distributed, electronic research network to help patients and their doctors determine which treatments work best for specific diseases and conditions

WASHINGTON (August 11, 2008) – The Joseph H. Kanter Family Foundation and the eHealth Initiative Foundation (eHI) announced the launch of the Partnership for Connecting for Research on Outcomes and Effectiveness, a national effort that will create a model for using electronic health information from multiple data sources—including electronic health records--while protecting patient privacy, to offer unbiased, evidence-based guidance on what treatments work best—vital information that can improve quality and safety and drive down costs in the health care system.

Developing a nationwide public-private partnership to use electronic data for outcomes research on what works in health care has been a long-time goal for Joseph H. Kanter, chairman of the Joseph H. Kanter Family Foundation. “This Partnership represents the culmination of what was started when former Senate Majority Leaders Bill Frist (R-TN) and George Mitchell (D-ME) joined me at the Newsmaker Breakfast at the National Press Club in a bipartisan call for a system that will help patients and their doctors understand how to better treat disease,” said Kanter, who is also serving as chairman of the Partnership. “Through this Partnership, we will accelerate research efforts by creating the first major pilot of a distributed research network using personal electronic health records to improve the quality and cost-effectiveness of health care for all Americans.”

The need for independent outcomes and comparative effectiveness research has been trumpeted by many public and private sector leaders, including our presidential candidates, members of Congress, the Congressional Budget Office, the Institute of Medicine, the Medicare Payment Advisory Commission, the Agency for Healthcare Research and Quality, and leaders from across every sector of health care. Just this month, U.S. Senate Finance Committee Chairman Max Baucus (D-MT) and Senate Budget Committee Chairman Kent Conrad (D-ND) introduced a bill to create a private Health Care Comparative Effectiveness Institute that would be governed by a public-private sector Board.

Although the ongoing policy discussions and the introduction of new legislation are both important steps forward, an important ingredient critical to the success of developing better evidence—a distributed, electronic network to develop better empirical evidence—has garnered little attention to date. The Partnership will work with experts and health care stakeholders to explore the organizational, technical, and policy aspects of using a distributed research network that leverages electronic health records to support the independent assessment of the clinical effectiveness of various treatments. This process will also provide more concrete guidance on the different kinds of evidence needed to identify “what works best” in health care: understanding how diseases progress for particular types of patients, identifying safety problems, comparing the benefits and risks of treatment alternatives, and comparing different medical practices that may affect quality and cost—and the reforms that can affect these practices. The Partnership looks forward to working with members of Congress and other policymakers to develop a common path forward that will improve health care for all Americans.

The Joseph H. Kanter Family Foundation has engaged Mark B. McClellan, MD, PhD, director of the Brookings Institution Engelberg Center for Health Care Reform, former administrator of the Centers for Medicare and Medicaid Services and former commissioner of the Food and Drug Administration, to serve as an advisor to the effort.  “This new partnership is a critical step for moving from the promise to the reality of better evidence that can improve health care,” said McClellan. “This Partnership is a practically-grounded strategy to determine how to leverage electronic health information—such as electronic health records—to help answer key questions about how health policy reforms can promote more effective, personalized care.”

The eHealth Initiative Foundation, an independent, non-profit, multi-stakeholder organization whose mission is to improve the quality, safety and efficiency of health care through information and information technology, will staff and coordinate the activities of the Partnership. This collaborative effort will draw upon eHI’s members as well as others representing academic and research institutions, clinicians, consumer and patient groups, employers, health plans, hospitals and other providers, laboratories, the life sciences industry, pharmacies, public health agencies, and state and local leaders, as well as nationally recognized experts in outcomes research, informatics, and privacy to lend their support in the collaborative development of methods and strategies. The Partnership will also build upon the experience eHI has gained through its Connecting for Drug Safety Collaboration, which is currently developing and piloting methods for using clinical and claims data to support post-market surveillance and drug safety.

"We are delighted to be partnering with the Joseph H. Kanter Family Foundation on this important initiative,” said Janet M. Marchibroda, eHealth Initiative chief executive officer. “Health IT and the responsible use of anonymized electronic health information promise to considerably advance the quality, safety and effectiveness of health care in our country, not only by providing significant support to research on better evidence, but also by bringing the results of that research to doctors and their patients to help them make better health care decisions.”

Over an 18-month period, the Partnership will support research to improve evidence on medical care, including the comparative effectiveness of alternative treatments and strategies for delivering care, by developing:

• A summary and analysis of current methods for using electronic health information to support research on outcomes and effectiveness.
• An assessment of the different data types available for such research, including but not limited to, clinical information which resides in electronic systems implemented primarily for care delivery, claims data, federal data systems, and information voluntarily provided by consumers for research—with their consent, through personal health records or other consumer-facing applications.
• Consensus policies developed collaboratively with consumers and patients, providers, payers and researchers that will effectively address privacy and confidentiality concerns and build and maintain public trust for using electronic health information for research on outcomes and effectiveness.
• A prototype distributed research network that tests and evaluates methods for using electronic health information for developing different kinds of evidence, while effectively protecting patient privacy.
• A set of recommendations for the organization, governance, sustainability, privacy, and technical aspects of an operational distributed network to support outcomes and effectiveness research. 

About the Joseph H. Kanter Family Foundation

The Joseph H. Kanter Family Foundation has played a critical leadership role in raising awareness of and advancing the creation of a National Health Outcomes Database, the research findings of which can be easily understood and accessed by patients and doctors. The Kanter Foundation’s “Health Legacy Partnership” includes more than 300 partners including nationally recognized leaders, corporations, federal agencies, national health foundations, and stakeholders from every sector of health care. For more information, go to www.healthlegacy.org.

About eHealth Initiative and its Foundation

The eHealth Initiative and its Foundation are independent, non-profit affiliated organizations whose missions are the same: to drive improvements in the quality, safety, and efficiency of health care through information and information technology. eHI engages multiple stakeholders, including clinicians, consumer and patient groups, employers, health plans, health IT suppliers, hospitals and other providers, laboratories, pharmaceutical and medical device manufacturers, pharmacies, public health, public sector agencies, and its growing coalition of more than 200 state, regional and community-based collaboratives, to reach consensus on and drive the adoption of common principles, policies, strategies and actions that improve health and health care through information technology that is responsible, practical, sustainable, and builds and maintains the public’s trust.  For more information, go to www.ehealthinitiative.org.  

  New Page

Landscaping Outcomes Research and Comparative Effectiveness Analysis Issues

The eHealth Initiative Foundation’s Partnership for Connecting Communities for Better Health Care Programs is pursuing a number of exciting and important efforts. The Collaboration for Outcomes Research and Comparative Effectiveness Analysis is one of these efforts. The Collaboration is beginning a process to landscape the issues surrounding outcomes research and comparative effectiveness analysis.  We look forward to working both within the Partnership and with other parties on this initial effort.  Part of this effort is to ask the following questions:

1)  What are outcomes research and comparative effectiveness analysis?

2)  What basic scientific and technical issues are associated with this research?

3)  What basic legal and policy issues are associated with this activity?

4)  What steps would enhance opportunities for and the value of this research?

We believe answers will become more complete and understandable as we go through this landscaping process.  We have begun to place some content as partial answers to some of these questions and will provide more as work continues.

  

   New Page

  What are outcomes research and comparative effectiveness analysis

1. Outcomes Research 
2. Comparative Effectiveness Analysis 
3. Other Types of Research 
4. Organizations Doing Work In these Areas 
5. Relevant Types of Databases 
6. How Information is Utilized 
7. Additional Definitions 

  

1. What is Outcomes Research?

Among other things both the Agency for Health Research Quality and the Outcomes Research Consortium state:

• Outcomes research seeks to understand the end results of particular health care practices and interventions. End results include effects that people experience and care about, such as change in the ability to function. In particular, for individuals with chronic conditions—where cure is not always possible—end results include quality of life as well as mortality. By linking the care people get to the outcomes they experience, outcomes research has become the key to developing better ways to monitor and improve the quality of care.

The International Society for Pharmacoeconomics and Outcomes Research states:

• Outcomes research is the scientific discipline that evaluates the effect of health care interventions on patient health status, often using the ECHO model involving economic, clinical, or humanistic outcomes. Outcomes research is generally based on the conceptual framework that evaluation of treatment alternatives involves the simultaneous assessment of multiple types of outcomes that are often disease-related.

In 1994 the Foundation for Health Services Research provided a document called Health Outcomes Research:  A Primer.  This document stated, among other things:

• Outcomes research studies the end results of medical care – the effect of the health care process on the health and well-being of patients and populations. It spans a broad spectrum of issues from studies evaluating the effectiveness of a particular medical or surgical procedure to examinations of the impact of insurance status or reimbursement policies on the outcomes of care. It also ranges from the development and use of tools to measure health status to analyses of the best way to disseminate the results of outcomes research to physicians or consumers to encourage behavior change.
•  A hallmark of outcomes research is the breadth of issues it addresses. Outcomes research touches all aspects of health care delivery, from the clinical encounter itself to questions of the organization, financing and regulation of the health care system. Each of these factors plays a role in the outcome of care, or the ultimate health status of the patient. Understanding how they interact requires collaboration among a broad range of health services researchers, such as physicians and nurses, economists, sociologists, political scientists, operations researchers, biostatisticians and epidemiologists.
• Outcomes research evaluates the results of the health care process in the real-life world of the doctor’s office, hospital, health clinic and even the home. This contrasts with traditional randomized controlled studies… which test the success of treatments in controlled environments….
• Traditionally, studies have measured health status, or health outcomes, in terms of physiological measurements – through laboratory test results, complication rates (e.g.infections) or death. These measures alone do not adequately capture health status. A patient’s functional status, well-being, and satisfaction with care must compliment the traditional measures.

The Center for Outcomes and Effectiveness Research at AHRQ provides for a comprehensive approach to health outcomes research that includes:

• Patient-perceived health outcomes (functional status, quality of life, return to work)
• Clinical outcomes (mortality, morbidity, clinical quality indicators)
• Costs and utilization (length of stay, costs, resource utilization)
• Patient satisfaction (hospital, physician, overall satisfaction)

What is Comparative Effectiveness Research/ Analysis?

In their December 2007 Document, styled Research on the Comparative Effectiveness of
Medical Treatments: Issues and Options for an Expanded Federal Role, the Congressional Budget Office stated:

• As applied in the health care sector, an analysis of comparative effectiveness is simply a rigorous evaluation of the impact of different options that are available for treating a given medical condition for a particular set of patients. Such a study may compare similar treatments, such as competing drugs, or it may analyze very different approaches, such as surgery and drug therapy. The analysis may focus only on the relative medical benefits and risks of each option, or it may also weigh both the costs and the benefits of those options. In some cases, a given treatment may prove to be more effective clinically or more cost-effective for a broad range of patients, but frequently a key issue is determining which specific types of patients would benefit most from it. Related terms include cost–benefit analysis, technology assessment, and evidence-based medicine, although the latter concepts do not ordinarily take costs into account.

In October 2007 Congressional Research Service provided a Report to Congress called  “Comparative Clinical Effectiveness and Cost-Effectiveness Research:  Background, History, and Review.”  This report stated:

•  Comparative effectiveness research is a term that has been defined by people in many different ways. All agree that comparative effectiveness research compares the effectiveness of two or more health care services or treatments, and is one form of health technology assessment. It compares outcomes resulting from different treatments or services, and provides information about the relative effectiveness of treatments.
•  Additional specifics about the research and its definition are sources of contention. In particular: (A) Effectiveness — How should effectiveness be measured? Should the research compare only the effectiveness (the effect in routine clinical practice) or also the efficacy (the effect under optimal conditions) of treatments or services? (B) Costs — Should costs be included in the research? Should the costs be reported separately from the effectiveness results? Or should a cost-effectiveness ratio be the ultimate goal?

 In June 2007 the Medicare Payment Advisory Commission (Medpac) issued a Report to Congress on Promoting Greater Efficiency in Medicare. Medpac stated:

•  Comparative-effectiveness analysis compares the relative value of drugs, devices, diagnostic and surgical procedures, diagnostic tests, and medical services. By value, we mean the clinical effectiveness  of a service compared with its alternatives. Comparative-effectiveness information has the potential to promote care of higher value and quality in the public and private sectors. Comparative information would help patients and providers become better informed and make value-based decisions. Most public payers—including Medicare—and private payers do not encourage patients or providers to consider the value of a service when making health care decisions. Information about the value of alternative health strategies might improve quality and reduce variation in practice styles. Use of comparative effectiveness research might improve health but will not necessarily reduce spending. Many effective treatments are underused, and effectiveness research might encourage their greater and more appropriate use (McGlynn et al. 2003). On the other hand, comparative-effectiveness research might reduce spending if, among a set of clinically comparable services, less costly services replace more costly services.

 In 2007 Institute of Medicine issued work styled “Learning What Works Best: The Nation’s Need for Evidence on Comparative Effectiveness in Health Care.”  That work states:

•  Within the overall umbrella of clinical effectiveness research, the most practical need is for studies of comparative effectiveness, the comparison of one diagnostic or treatment option to one or more others. In this respect, primary comparative effectiveness research involves the direct generation of clinical information on the relative merits or outcomes of one intervention in comparison to one or more others. Secondary comparative effectiveness research involves the synthesis of primary studies (usually multiple) to allow conclusions to be drawn. Secondary comparisons of the relative merits of different diagnostic or treatment interventions can be done through collective analysis of the results of multiple head-to-head studies, or indirectly, in which the treatment options have not been directly compared to each other in a clinical evaluation but reside in larger databases. Conclusions utilize inferential adjustments based on the relative effect of each intervention to a specific comparison, often a placebo.

 Who is doing work in this area?

 

 Private Sector

 The 2007 CBO Report lists a number of private sector activities in this area:

• Several private-sector organizations exist primarily or exclusively to assess medical treatments and technologies. One prominent example is the Technology Evaluation Center that is part of the Blue Cross Blue Shield Association. Its analyses are based on systematic reviews of the available literature and therefore rely on clinical trials or other studies that have already been conducted. (In such reviews, more weight is given to studies that are judged to be of higher methodological quality.) The center produces about 20 to 25 new assessments of drugs, devices, and other technologies each year; the analyses consider clinical effectiveness but generally do not assess cost-effectiveness.  
• For-profit private-sector firms that specialize in technology assessments represent another source of analysis. Hayes, Inc., is one of the larger firms in the field. Such firms also conduct systematic reviews and evaluate medical and surgical procedures, drugs, and devices in return for a fee or on a subscription basis. Organizations that are similar but operate as nonprofit entities—sometimes affiliated with academic or medical centers—include the ECRI Institute and the Tufts-New England Medical Center’s Cost-Effectiveness Analysis Registry (which provides an extensive list of the cost-effectiveness ratios that are available from published studies).
• In addition, private health plans—most commonly, larger or more integrated ones—conduct their own reviews of evidence and sometimes undertake new analyses of comparative effectiveness using claims data for their enrollees. Health plans may choose to publicize the results, or they may decide to keep their findings confidential and use them to shape their policies regarding coverage of and payment for the treatments in question. For example, health plans usually have an entity known as a pharmacy and therapeutic committee that considers the evidence regarding the relative effectiveness of different prescription drugs and makes recommendations about which ones should be covered (that is, included on formularies) or given preferred status. An example of a more public and collaborative effort is the HMO Research Network, a consortium of more than a dozen health maintenance organizations from different parts of the country; started in the mid-1990s, it brings together researchers to share findings and, in some cases, uses data from several plans as the basis for analysis.

     Federal Programs

The 2007 CBO Report also lists a number of Federal Programs:

• More recently, the Agency for Health Care Research and Quality (AHRQ) has been the most prominent federal agency supporting various types of research on the comparative effectiveness of medical treatments. Established in 1989 as the Agency for Health Care Policy and Research, AHRQ is an arm of the Department of Health and Human Services (HHS).  It currently has a staff of about 300 and an annual budget of over $300 million, which primarily funds research grants to and contracts with universities and other research organizations covering a wide range of topics in health services.
• AHRQ has undertake a number of intiatives related to comparative effectiveness. One such step—initially taken in collaboration with the American Medical Association and America’s Health Insurance Plans, a coalition of insurance companies—has been the creation of a national clearinghouse for treatment guidelines, which are designed to summarize the available medical evidence on the appropriate treatments for various conditions. AHRQ has also endorsed about a dozen evidence-based practice centers around the country. Generally affiliated with a university, those centers analyze and synthesize existing evidence about treatments and technologies. Although many studies sponsored by AHRQ have examined only the relative clinical benefits of different treatments, some have also analyzed their cost-effectiveness. Research on comparative effectiveness has accounted for only a modest portion of AHRQ’s budget, though.
• Most recently, section 1013 of the Medicare Modernization Act of 2003 authorized AHRQ to spend up to $50 million in 2004 and additional amounts in future years to conduct and support research with a focus on "outcomes, comparative clinical effectiveness, and appropriateness of health care items and services (including prescription drugs)" for Medicare and Medicaid enrollees. The actual funding appropriated for that initiative has been $15 million per year. Using that funding, AHRQ has established an "Effective Health Care" program consisting of three main functions: reviewing and synthesizing existing evidence (using its evidence-based practice centers); generating new information using a set of approved research centers (such as the HMO Research Network) that have access to data from medical claims and electronic medical records; and publishing findings in formats that are geared to the differing needs of clinicians, patients, and policymakers.
• Other federal agencies also engage in various activities related to comparative effectiveness research—efforts that receive less attention than AHRQ’s activities but that are probably larger in dollar terms. The Department of Veterans Affairs (VA) has a very substantial research program that reviews evidence from the medical records of its patients, focusing particularly on the clinical effectiveness of treatments. The department also sponsors evidence reviews through a technology assessment program and helps fund clinical trials—including the study comparing stents to drug therapy mentioned above. Indeed, over the past 30 years, some of the most influential clinical trials have been supported by and conducted in the VA health system, including the first major trials that demonstrated the value of bypass surgery over medical therapy for some forms of coronary artery disease as well as head-to-head studies of drugs that treat prostate enlargement.
• Another source is the National Institutes of Health (NIH), part of HHS, which is the leading federal sponsor of medical research—primarily in the form of clinical trials. Although comparative effectiveness is not a focus of that research, over the years NIH has sponsored a number of trials that compare treatments directly.
• The Centers for Medicare and Medicaid Services (CMS) has helped to sponsor a limited amount of research on comparative effectiveness (for example, it covered the medical costs of the study of lung-volume-reduction surgery). When making decisions about what services are covered, however, CMS generally considers only whether devices and procedures are clinically effective. It has sponsored some studies comparing the effectiveness of different treatments but has done so largely to determine whether to establish separate payment rates for similar treatments. For example, CMS is currently cosponsoring a trial with NIH that may eventually compare the effects of daily dialysis for kidney patients with the conventional treatment of dialysis three times per week. If daily dialysis proves more effective for certain patients, CMS could modify its payment policy to cover the additional costs of more frequent treatment for those patients.

          Amount Spent in the U.S.

• Estimating the total amount that is spent in the United States each year on research that compares the effectiveness of medical treatments is difficult. According to one recent analysis, the federal government spent about $1.5 billion in 2005 on all health services research, a broader category that includes some of the work on comparative effectiveness but also encompasses many other types of studies. Estimating private expenditures is even more challenging. Although drug and device manufacturers spend billions of dollars each year on clinical trials aimed at demonstrating the safety and efficacy of new products, the vast majority of those efforts contribute to comparisons of treatments only indirectly. Data are simply not available on how much is spent by private organizations such a health plans, medical specialty societies, and technology assessment centers to compare medical treatments and procedures. Nevertheless, one recent study estimated that less that $2 billion is spent annually on comparative effectiveness research in this country—and even that rough estimate is subject to uncertainty

Other Countries

From 2007 CBO Report:

• Other developed countries also face challenges financing health care costs and have taken various steps to assess the comparative effectiveness of treatments. Unlike the United States, many of those countries establish overall budgets for their national health systems and regularly use the data on comparative effectiveness that are available to help determine the treatments and procedures to be covered and, in some cases, the payment rates. Despite differences in other countries’ health insurance systems, the approaches that they have taken to organizing and funding those research and review activities could have lessons for any increased U.S. efforts.
• Perhaps the best known example of an agency that assesses comparative effectiveness is the National Institute for Health and Clinical Excellence (NICE), which was established in 1999 as part of the United Kingdom’s National Health Service (NHS). It analyzes both the clinical effectiveness and cost-effectiveness of new and existing medicines, procedures, and other technologies and provides guidance on appropriate treatments for specific diseases or types of patients. To date, NICE has published appraisals of over 100 specific technologies, guidance on the use of about 250 medical procedures, and about 60 sets of treatment guidelines—a substantial but not exhaustive list. If NICE approves a drug, device, or procedure, it must be covered by the NHS, but local health authorities make coverage decisions about treatments that NICE has not yet evaluated. With a staff of about 200 and an annual budget of about 30 million pounds (roughly $60 million), NICE does not fund new clinical trials or other forms of primary data collection. Instead, it commissions systematic reviews of existing research on clinical effectiveness and combines those findings with models of cost-effectiveness. Clinical trials are funded by the British Ministry of Health but (as in this country) data on total spending in the United Kingdom for research on comparative effectiveness are hard to come by.
• Other countries such as Australia, Canada, France, and Germany have similar review processes, though the organizational and financing arrangements vary—and in several cases, the structures have recently been changed. For example, France established a new agency in 2004 to bring together a number of related activities, including the evaluation of drugs, devices, and procedures, publication of clinical guidelines, accreditation of providers, and dissemination of medical information. Germany established a new agency in 2000 that conducts technology assessments and a new Institute for Quality and Efficiency in 2004 that evaluates health care services. Discussions about the use of comparative effectiveness in those countries sometimes focuses on their review processes for prescription drugs, but their efforts generally encompass all forms of acute medical care. (For all the attention they receive, drug costs represent less than 15 percent of health care spending in the United States—so research that focused only on medications would miss the vast majority of services and would not be able to compare drug therapy with surgical procedures or other interventions.)
• Although those countries all have government-run health care systems, they have taken different approaches regarding the placement of and funding for their assessment bodies. In the United Kingdom and Australia, the agencies are part of the government’s health departments; France and Canada have established independent not-for-profit organizations; and Germany has taken a mixed approach (the Institute for Quality and Efficiency is independent, but the technology assessment agency is an arm of the health ministry). Financing arrangements vary correspondingly: Funding in the United Kingdom and Australia comes from their health departments, whereas Germany’s independent institute is funded by a levy on inpatient and outpatient health care services (which are mainly reimbursed by the country’s regional health insurance funds), and the French agency gets its funding from a combination of taxes on promotional spending by drug companies, government subsidies, and accreditation fees. Health ministries in Australia, Canada, France, and Germany also help fund clinical trials and other forms of primary research, but total spending related to comparative effectiveness in those countries is also difficult to estimate.
• Given the interest that has developed in many countries, it is not surprising that several international organizations have become involved in comparative effectiveness research. The best known may be the Cochrane Collaboration—a nonprofit organization that has a network of volunteers who conduct systematic reviews of treatments. Many of its activities are organized through centers located around the world, including one in the United States. Founded in 1993, the Cochrane Collaboration maintains an accessible database that now contains more than 4,500 reviews; its limited funding comes primarily from subscription fees for its quarterly journal. Any new or expanded U.S. entity that would organize and fund research on comparative effectiveness would probably draw upon Cochrane’s findings and the results of research conducted in other countries (to the extent such research was applicable to U.S. patients).

Relevant Types of Health Care Databases

 

Encounter Data

Encounter datasets represent databases whose purpose is to maintain a record of health care encounters, typically maintainedby payers, to track reimbursement. Data are typically groupedby ambulatory care visit or hospital admission. However, thesedata can be summed to generate data on a region, and, if uniquepatient identifiers are available, can also be linked to createa longitudinal history for individual patients. Encounter datasetscan be extremely large. For example, the Medicare Provider Analysisand Review (MEDPAR) file contains records for 100% of Medicarebeneficiaries who use hospital inpatient services. The nationalfile consists of approximately 11 millionrecords.

There is considerable variation in the quality and detail of these datasets. For example, datasets may allow different numberof diagnoses or procedure codes, and have varying detail regardingutilization. When data are collected for billing purposes,financial incentives driving the data collection can impair itsvalue. Datasets coded by medical records departmentssuffer from the vagueness of the International Classificationof Diseases system and the errors inherent in the abstractionprocess. Analyses based on variables available in encounterdata may miss important relationships present in detailed clinicaldata).

Enrollment Data

In addition to information about patients' encounters with the medical system, one might also wish to know about the denominatorpopulation from which the encounter numerator is drawn. This allowsthe investigator to draw inferences about population incidenceand utilization for various diseases. Census data can provideage-, race-, and gender-specific data as well as some informationon socioeconomic variables at the national and regional level.Other denominator sources, including Health Maintenance Organizationenrollment databases, are available for selective analyses. Inthe United States, the Health Care Financing Administration hasdata on Medicare enrollees organized by state, county, and zipcode. Investigators who are limiting their analysis to a specificinsurer or managed care organization can often obtain enrollmentdata on the insured population. These data are limited by transferbetween insurance plans, which makes estimating a stable denominatorfor any given year achallenge.

Electronic Clinical Data

The electronic medical record promises to be a boon to outcomes research should a usable format gain widespread acceptance. Whether or not a formal electronicmedical record exists, virtually all clinical information in themodern hospital, including laboratory data, pharmacy, and manydiagnostic test results are converted to electronic form. Theextent to which data from these separate systems can be collatedinto a usable resource and subsequently analyzed for researchpurposes has not been explored. Barriers to the effective useof hospital databases include their sheer volume, the lack ofstandardized data collection and computer formats, and protectionof confidentiality and proprietaryinterests.

Data Registries

Data registries are databases focused on a particular disease or intervention.  Data registries have the advantage of collectinga set of predefined data on a large number of patients with homogenouscharacteristics. Frequently, data registries rely on collectionfrom a single or several academic institutions and therefore maynot generalize to other clinical settings. Because they only includepatients with the disease or treatment, many important questionsabout incidence, variability in management, and comparative outcomecannot beanswered.

Performance Data

In the current era of health care reform, several organizations and agencies have begun to collect data explicitly for thepurpose of assessing institutional and provider performance. These datasets are of great potentialvalue in that they are collected across a number of institutionsand usually contain data significantly richer in clinical detailthan other encounterdatasets.

Survey Data

Traditionally, assessments of therapies have been based on clinical measures and objectiveoutcomes such as mortality. Increasingly, however, attention isbeing focused on the importance of patient-centered outcomes.Information on such outcomes is available through surveys of patientpreferences and satisfaction with care.

Clinical Trial Data

Randomized controlled trials contain a considerable amount of detailed information on the demographics, severity of illness,and outcomes of critically ill patients that has been collectedprospectively under rigorous conditions. In addition to meta-analysis,which combines results from several RCTs to estimate the treatmenteffect, it may be possible to perform a secondary cohort analysison the patients in clinical trials. While these datasets seem promising, the same factorsthat limit the assessment of effectiveness by RCTs may make extrapolationfrom their patient data to broader populations problematic. Consequently,readers and researchers should consider carefully the applicabilityof these data sources for the potential questionsasked.

Prospective Cohort Data

There are several datasets that were collected with the specific intention to be representative of specific types of populations. The principal limitations governing their use include limitedaccessibility to investigators and the data elements collected.

The above examplse are from Am. J. Respir. Crit. Care Med., Volume 160, Number 1, July 1999, 358-367 AMERICAN THORACIC SOCIETY
Outcomes Research in Critical Care
Results of the American Thoracic Society Critical Care Assembly Workshop on Outcomes Research GORDON D. RUBENFELD, DEREK C. ANGUS, MICHAEL R. PINSKY, J. RANDALL CURTIS, ALFRED F. CONNORS Jr., GORDON R. BERNARD, and The Members of the Outcomes Research Workshop

How is Information Utilized

AHRQ Discussion of Levels of Impact Resulting from Outcomes Studies

Level I—Impact on Further Research. This level includes effects of research studies that do not represent a direct change in policy or practice. This includes new tools and methods for research, instruments and techniques to assist clinical decisionmaking and studies that identify areas in which scientific knowledge is absent but needed. Level I impacts are also provided when studies describe findings that are inconsistent with current clinical paradigms and stimulate rethinking and questioning within a clinical specialty.

Level II—Impact on Policies. A policy or program is created as a direct result of the research, e.g. the information is used by health plans, public programs such as Medicaid, professional organizations, legislative bodies, regulators, and/or accrediting organizations.

Level III—Impact on Clinical Practice. The research results in a change in what clinicians or patients do, or in changes in a pattern of care. These changes may be demonstrated in a limited study population as a result of a specific intervention, or they may be trends identified outside a formal research context.

Level IV—Impact on Health Outcomes. This includes actual impact on health outcomes including those that are clinical, economic, related to quality of life, or related to satisfaction. They may be demonstrated in a limited study population as a result of a specific intervention, or beyond and outside of a formal research context.

Additional Definitions

The following definitions were compiled in a presentation dated October 25, 2007 by Ted Buckley, Phd, Director of Economic Policy at the Biotechnology Industry Organization. The definitions were compiled from the following sources:

• Academy Health. Glossary of Terms Commonly Used in Health Care 2004 edition.
• Berger, Marc L. Health Care Cost, Quality and Outcomes: ISPOR Book of Terms International Society for Pharmacoeconomics and Outcomes Research. 2003.
• Drummond, Michael F. et al. Methods for Economic Evaluation of Health Care Programmes 2nd edition. Oxford University Press. 1997.
• Gold, Marthe R. et al. editors. Cost-Effectiveness in Health and Medicine Oxford University Press. 1996.

Contingent Valuation Determining an individual’s maximum willingness to pay for a good or service that is not available in the marketplace. The determination is often made through hypothetical survey questions. Drawbacks include need for a large sample size and starting point bias.

Cost-Benefit Analysis Derived from economic theory, this analytical technique lists and compares the net costs and net benefits of a health care intervention. Both the net costs and net benefits must be expressed in monetary units. In order to value the net benefits, the improved health outcomes must be expressed in monetary units. A variety of techniques can be used to value improved health outcomes, one of which is willingness to pay. The bottom line of the analysis is net benefit. It may be used to compare a treatment to a placebo or to another treatment, or even to compare two treatments for unrelated conditions. A key problem with the methodology is the conversion of health improvements into monetary units. Many of these analyses require significant assumptions in the models that are employed.

Cost-Comparison Analysis Compare only the costs associated with two or more health care treatments. There is no inclusion of health benefits in the analysis. As with all accounting of costs, care must be taken to include all costs. Uncertainty exists in determining the proper allocation of certain costs (e.g., one time, shared and fixed costs.)

Cost-Consequence Analysis A comparison of alternative health interventions in which the outcomes and costs are listed without aggregating the results. Whereas, the cost-benefit ratio results in a single aggregated result, the cost-consequence analysis does not. There is no prescribed weighting system to indicate the relative importance of different benefits or costs. Thus, while this analysis avoids the problems of the conversion of different outcomes into a common metric, it leaves the comparison of outcomes that may be difficult to compare to the decision maker.

Cost-Effectiveness Analysis A tool in which the effects and costs of a health intervention are compared to a placebo or to another treatment, or even to compare two treatments for unrelated conditions. The effects are health outcomes and must be expressed in the same units (e.g., life years gained); however, they are not expressed in monetary units as in cost-benefit analysis. If there are just two alternatives being compared, a comparison is made by dividing the difference in cost of the two treatments by the difference in outcomes. As with the cost-benefit analysis, the conversion of health improvements (outcomes) into a common metric can be quite challenging (e.g., one treatment increases the quality of life but not survival time, but another treatment increases survival time.)

Cost-Identification Analysis Identification of all relative costs and their importance.

Cost-of-Illness Study A study to determine the total cost, including treatment costs, of a disease or health condition on society.

Cost-Minimization Analysis When the outcomes are equivalent, this tool is used to compare the net costs between different health interventions. Because the outcomes must be equivalent, the value and use of this technique is very limited for assessing new medical interventions.

Cost-Utility Analysis A comparison of different health interventions where the health outcomes are translated into units of utility (e.g., QALYs.) The analysis is expressed in terms of a ratio of the incremental cost of the two alternatives to the incremental health effects of the two alternatives. The result of the analysis is the “cost/QALY” of the intervention. It is used to determine the relative value of alternative health interventions. As with the cost-benefit and cost-effectiveness analyses, the conversion of health improvements into a common metric can be quite challenging. Note: some consider the cost-utility analysis to be a special type of cost-effectiveness. The three terms above all have the same challenge of finding a common metric that describes the change in health status associated with the treatment options that are being evaluated. Terms to describe health outcomes range from very specific measures (e.g.,symptom free days) to more abstract terms such as life years gained, healthy life years gained, QALYs etc. Differences in the description of outcomes may further impair the ability to condense various studies into one common health outcome result.

Effectiveness Represents outcomes achieved from a treatment or health intervention in real, practical settings (e.g., the real world)

Efficacy Represents outcomes achieved from a treatment or health intervention under ideal circumstances (e.g.,clinical trials)

Evidence-Based Medicine Based on systematic review of all available data usually in published domain or available from organizations, the identification of best evidence to inform decision making about the care of individual patients. The information may also be utilized with same or different conclusions to describe best evidence to inform decision making about the care of populations. Evidence-based medicine (EBM) requires that physicians have access to critical, unbiased reviews of all currently available information. The goal is to enable physicians to bring unbiased sources of information into the patient encounter and use them in the clinical decision-making process. EBM will take into account specific patient characteristics and preferences. Note: many consider comparative effectiveness to be a type of EBM.

Evidence Synthesis A meta-analysis – also referred to as secondary clinical effectiveness research – which is a structured assessment of evidence from multiple primary studies to develop a conclusion.

Health Technology Assessment Also known as HTA. It is an evaluation that examines the effects and impacts of health care technology or treatment. The effects and impacts are broad and can include, but are not limited to, safety, efficacy, effectiveness, economic, political and ethical. It is intended to educate decision makers as to the direct and indirect consequences of a given technology of treatment.

Primary Clinical Effectiveness Research Design and implementation of structured research protocols to produce data on the results of one or more diagnostic or therapeutic interventions of interest. The evaluation can measure either effectiveness or efficacy.

Quality-Adjusted Life Year Combines gains or losses in both quantity of life (mortality) and quality of life (morbidity) into a single measure that is years of life saved by a health intervention, adjusted according to the quality of those years. The adjustment is made according to some evaluative measure. Typically, the range is from 0 to 1 where 0 is death and 1 is optimal health. Thus a year of life in non-optimal health would be rated somewhere between 0 and 1. The use of Quality-Adjusted Life Years (QALYs) enables comparison across diseases and programs. The precise score given to different levels of non-health are based on tools such as utility measures (e.g.,standard gamble), Health Utilities Index, EuroQolEQ-5D. QALYs can be used in studies with or without an economic component. There are a variety of issues associated with the conversion of health status into QALYs. The first is that conversion of certain health states must be based on preferences. However, whose preferences should they be based upon - individual patient preferences, preferences of an informed public, or some other group? For instance the informed public might value the quality of life of a year in a wheelchair very differently than a person who is in a wheelchair. Another issue is that different tools may give systematically different results. The QALY does not differentiate whether a gain (loss) in QALYs comes from a small gain (loss) to a large number of people or a large gain (loss) to a small number of people.

Randomized Control Trial A clinical trial in which subjects are randomly assigned to different treatment groups. The research study is well described with controls to exclude potential sources of systematic bias or confounding factors – examples include the use of inclusion or exclusion criteria, use of blinded populations so investigator or analytical bias does not influence results. The randomized control trial is an effort to identify the most robust knowledge about the treatment groups without systematic bias or confounding factors.

Retrospective Analysis Analysis based on data that is currently available (i.e., that has been already generated). Generally, the data comes from sources such as insurance claims data or hospital discharge data. These data are usually derived from health care claims used to secure payment in health care systems. For that reason, their primary goal is to insure payment and not research on patients as an outcome. Because the data is usually not designed with research in mind, the data quality may be lacking important clinical data.

Willingness to Pay An estimation of the maximum dollar amount an individual would pay to obtain a good, service or reduction in risk. Willingness to Pay (WTP) may be used in cost-benefit analysis to determine how much one is willing to pay for a certain health outcome. The estimation is made in order to measure the value of the good, service or reduction in risk to the individual. There are several issues associated with using WTP. An individual’s ability to pay constrains his or her WTP. Therefore, if there are health care treatments for diseases that affect primarily the wealthy, those outcomes and treatments will receive higher WTP than health care treatments for diseases that affect primarily the poor. If open ended questions are used to ascertain individuals WTP, the results will vary widely and include many non-responses. Alternatively, closed ended questions may cause starting point biases.

 

 

 

 

From American Medical Information Association (AMIA) 2007 Secondary Data Conference

Taxonomy Working Group

6/8/2007

 

 

SECONDARY USES OF DATA

 

a.   Provide background data for system to implement real-time decision support for a specific patient based on repository of observations on similar patients (just-in-time outcomes research decision support with an N of 1)

b.   Protect and enhance public health

i. Report vital statistics

1. Report disease incidence and prevalence

2. Report disease outcomes and mortality

ii. Report infectious diseases

1. Export culture results

2. Export serology (antigen, antibody) results

3. Export DNA/RNA microbiology probe results

iii. Enable and support bio-surveillance

1. Detect sentinel events

2. Detect deviations from background rates

iv. Export data to health registries

1. Export data to cancer registries

2. Export data to rare disease databases

3. Send information to drug and device registries

v. Report wellness information

vi. Report toxic exposure

1. Report smoking exposure

2. Report exposure to secondary smoke

3. Report Agent Orange exposure

vii. Support population data analysis and reporting

1. Report biometric demographics

a. Weight

b. Height

c. Blood pressure measurements

d. Laboratory normal values

c. Conduct Health Research

i. Conduct clinical epidemiology and outcomes research

ii. Investigate disease natural history

iii. Investigate disease risk predictors

iv. Investigate treatment effectiveness

v. Investigate treatment response predictors

vi. Conduct health services research

vii. Analyze pharmacoeconomics

viii. Assess health technology

ix. Conduct health policy research

x. Support genetics research

1. Investigate correlations among diseases, phenotypes, and genotypesLink medical data to pedigree and family tree data

xi. Conduct clinical trials

1. Assess feasibility of research by determining availability of subjects and sites

a. Analyze patient data to optimize study design and protocol

2. Identify candidate subjects for research studies and protocols

d. Manage and improve the quality of patient care

i. Support quality management

1. Support staffing and resource management

2. Support outcomes management

3. Support acuity-driven management

ii. Improve patient safety

1. Support risk profiling

2. Detect and analyze adverse and sentinel events

3. Monitor and survey to prevent patient events

iii. Maintain pharmacovigilence

1. conduct post market drug and device surveillance

2. Report sales of OTC GI meds

e. Support quality assessment

i. Develop quality indicators

ii. Track quality indicators

iii. Support national quality reporting

1. Automate HEDIS reporting

f. Support and analyze financial activities

i. Automate claims processing

ii. Automate billing

iii. Support cost accounting

iv. Support activity based charge capture

v. Construct predictive models of costs and accounting

g. Create and test knowledge assets and decision support algorithms

i. Develop rules, protocols, alerts

ii. Develop order sets

iii. Test the efficacy of decision support rules

iv. Test the validity of computable knowledge 

v. Create and maintain terminology and representation formalisms

h. Develop security and confidentiality algorithms

i. Develop and test de-identification routines

i. Support market research

i. Target patient advertising, marketing, and sales

j. Track clinical training activities

i. Log number of successful procedures

ii. Log types of procedures performed

k. Credential health care providers

l. Detect illegal and inappropriate activity

i. Report drug screen results to detect illegal drug use

ii. Analyze prescribing patterns

iii. Detect fraud and abuse

1. Detect Medicare upcoding

iv. Export utilization profiles

m. Use patient data for patient-specific knowledge seeking

n. Use data to develop interoperability and interchange specifications

o. Use data for personal health management

i. Maintain personal health record

ii. Use own or family member’s health data to set goals, select and track health behaviors, and track progress toward goals

iii. Seek knowledge for managing own or family member’s health