Home Past IssuesAbout UsContact Us Twitter Icon Facebook Logo LinkedIn Logo
Dartmouth Medical School Dartmouth-Hitchcock Medical Center

Map Quest

The decades-long quest to parse the piecemeal nature of the American health-care system has put Dartmouth on the map, both medically and politically. But along with the plaudits and headlines has come criticism. Here, two of the experts behind the renowned Dartmouth Atlas of Health Care explain why its findings really do hold up.

Pursuing the trail
of the evidence

By Amos Esty

For more than 30 years, Dartmouth researchers have been investigating the causes and implications of regional variations in health care. The findings gathered at Dartmouth and published in the Dartmouth Atlas of Health Care had gained respect from the medical community and attention from the media long before the recent battle about health-care reform. But over the past year, with debate raging in Congress and in the press about how to improve the nation's health-care system, the work of Dartmouth Atlas researchers began to receive ever greater scrutiny from lawmakers, journalists, and the public. Peter Orszag, director of the Office of Management and Budget, used evidence from the Atlas to propose paying for expanded insurance coverage by cutting back on endemic waste in the health-care system. In June 2009, the New York Times reported that even President Obama was citing the Dartmouth research—discussing with aides a widely read New Yorker article by Dr. Atul Gawande that drew heavily on Atlas findings.

Esty is the managing editor of Dartmouth Medicine. Fisher is the director for population health and policy at the Dartmouth Institute for Health Policy and Clinical Practice (TDI) and a professor of medicine and of community and family medicine at DMS. And Skinner is the John Sloan Dickey Third Century Professor of Economics at Dartmouth College, as well as a professor of community and family medicine at DMS and at TDI.

Controversy over the research is likewise nothing new. In the 1970s, when DMS's Dr. John Wennberg began investigating regional variations in health care, he had trouble finding a journal that would accept his first article on differences in physician practice patterns. Over time, however, the conclusions reached by Wennberg and others at Dartmouth—that more-intensive care does not necessarily lead to better outcomes and that much of the money spent on health care is wasted—have become widely accepted.

But along with the Atlas's growing influence came louder criticisms. "Wrong Map for Health Reform" was the title of a critical Washington Post op-ed essay. "A Map to Bad Policy" was the title of another criticism of the Atlas, a commentary in the New England Journal of Medicine.

The criticisms have focused on a few major points. One is that spending more on patients does, in fact, lead to better outcomes. Another is that regional variations are caused primarily by differences in the health of patients or in socioeconomic factors, and that penalizing hospitals that spend more will hurt vulnerable populations. Critics also contend that the Atlas's conclusions—which are drawn from the Medicare claims database, the only comprehensive set of health-care records in the U.S.—are skewed by the fact that they're based on a patient population that's 65 and older.

Atlas articles

Read past Dartmouth Medicine articles about Dartmouth Atlas research.
Learn more

Atlas researchers have repeatedly responded to these and other criticisms, in peer-reviewed publications and in interviews with the lay press—as well as in the article that follows.

Identifying patterns amidst variation

By Elliott Fisher, M.D., and Jonathan Skinner, Ph.D.

The goal of the research underlying the Dartmouth Atlas of Health Care has been to improve our understanding of the causes and consequences of the well-documented variations in the way health care is delivered around the country. Thanks to funding from the National Institutes of Health and several not-for-profit foundations, notably the Robert Wood Johnson Foundation, we have made major progress on addressing such questions as these: Why do patients in some regions spend twice as much time in the hospital during their last six months of life than patients in other regions? Why was the rate of coronary stents three times higher in Elyria, Ohio, than in nearby Cleveland? And, most importantly, what do these differences mean for patients? The aim of the Dartmouth research is to advance the science of health care delivery to understand what actually happens to patients and what can be done to make care better. [See this article for more on the science of health care delivery.]

While many of the findings are broadly accepted, and their implications are already being translated into practice, confusion about the Dartmouth work remains—especially in the public arena, where it can be difficult to briefly explain a complex and nuanced body of research.

Nevertheless, we have tried to summarize and respond to some common points of confusion.

The Dartmouth Atlas shows a more than twofold variation in per-capita Medicare spending among different regions of the country. How much of it is just a consequence of the higher cost of living in places like San Francisco or New York?
The brief answer is that cost of living has little impact on overall variation in per-capita health-care expenditures. Ever since Dr. John Wennberg and Megan Cooper published the first Atlas in 1996, it has been clear that variation exists in many other utilization measures—hospital days, surgical rates, and physician visits, for example—that are not biased by the high prices in New York or the low prices in Oklahoma.

More recently, we have also provided price adjustments on our spending measures. For the most part, these adjustments made little difference. The highest two regions in spending, Miami, Fla., and McAllen, Tex., were still the highest after price adjustment.

The one exception was New York City. Prices are higher there for everything, including health care. But prices are uniquely high in New York because of generous Medicare payment policies that reward hospitals with very large medical resident training programs. With an estimated one-sixth of all residents training in New York, hospitals there receive more per hospital discharge than anywhere else in the country. They seem to keep patients in the hospital longer as well—so New York's hospital days and physician visits in the last two years of life are among the highest in the country.

But in general, adjusting for price differences leads to only a modest decline in overall variation in expenditures. It is utilization—the amount of care delivered to patients—that explains most of the regional variation in per-capita Medicare spending.

Regions where Medicare spends more often have many people below the federal poverty line. Doesn't poverty explain much of the geographic variation in spending?
No. In places like Miami, everyone, poor and rich alike, gets more health care. In a New England Journal of Medicine paper, we found that poverty explains little of the variation in spending—at most 4%; a more recent study found even less.

It is utilization—the amount of care delivered to patients—that explains most of the regional variation in per-capita Medicare spending. We found that poverty explains little of the variation in spending—at most 4%.

This may appear puzzling—why wouldn't people living in poverty have greater health needs? The answer is that they do. But many of the poor do not enjoy the same access to care that wealthier Medicare enrollees do. Nor is poverty limited to high-cost regions like Los Angeles. It is also present in low-cost regions such as Albany, N.Y., and Richmond, Va.

What does explain some regional variation—but not very much—are differences in the underlying health of the population. Residents of Mississippi and the Bronx do experience more adverse health events than those of Vermont, and it should be no surprise that regions with poor health should experience higher rates of Medicare expenditures. But these health differences do not explain the vast majority of regional variations in health-care expenditures or utilization. A recent study by the Medicare Payment Advisory Commission (MedPAC) suggested a larger role for illness in explaining regional variations. However, their adjustments suffer from a serious bias—that people are more likely to be "diagnosed" with a disease when their physician or hospital treats them more intensively. As we showed in a recent New England Journal of Medicine paper, this bias makes patients in high-intensity areas appear sicker than they really are.

The Dartmouth research has compared the care of patients during the last two years of their life at different hospitals, but doesn't that fail to take into account the possibility that some hospitals are better at preventing death?
Two questions are confounded here. The first question is whether end-of-life expenditures accurately predict how intensively hospitals also treat patients with conditions that are serious but not fatal, such as heart attacks. As we have shown, they do.

The second question is whether higher-intensity (and thus higher-spending) hospitals achieve better outcomes. Obviously, one can't answer this question just with a sample of people in the last two years of life, since they all die. Instead, the Dartmouth team has looked at this question by focusing on patients with several specific conditions, such as hip fractures and heart attacks, and followed them for several years to see how they fared. On average, higher spending was not associated with better outcomes.

But isn't there some evidence that spending more on care does, in fact, lead to better outcomes?
The key question is: Spending more on what? Dartmouth research comparing spending differences between both regions and hospitals has found that most of the spending gaps for the elderly were due to differences in the use of a hospital as the site of care (versus, say, a hospice, nursing home, or doctor's office), as well as to discretionary use of specialist visits and tests. Yet the higher spending on these services does not appear to offer any overall benefits. Other Dartmouth research, however, has suggested that hospitals that spend more on effective care do in fact get better outcomes.

But several other studies have found that some higher-spending hospitals do have better outcomes. One study looked at heart failure patients cared for at six California hospitals and found that patients in the most expensive hospitals had lower mortality. Using our own data, we found comparable results for these six hospitals—but when we looked at all California hospitals, the association between spending and outcomes no longer held up.

Other studies have shown small positive benefits associated with specific types of services. A study by our Dartmouth colleague Dr. David Goodman found that some regions probably have too few neonatologists, for example. Another study found better outcomes for Florida tourists admitted to more-intensive emergency rooms, but not for Florida residents.

But overall, most peer-reviewed studies have found a negative association between spending and outcomes. The key message from these studies is that many health systems are able to provide high-quality care at low cost, suggesting that we don't need to spend more to get better outcomes.

Is it possible to make reliable predictions about those under age 65 who are covered by private insurance by looking just at Medicare claims data?
The available evidence does suggest that hospitals and regions that provide more-intensive care to Medicare patients also provide more-intensive care for their non-Medicare patients.

We are slowly learning more about the under-65 population and how complicated and opaque local markets for private insurance can be. For example, in private insurance markets, prices per procedure can vary wildly across regions—so total per-capita regional spending in under-65 markets is often not associated with per-capita Medicare spending in those markets. There are a variety of theories about how hospitals might be shifting costs from the Medicare market to the private market and vice versa, but these are still somewhat speculative.

But suppose one accepts for the moment the critics' view that Medicare spending is not predictive of under-65 spending. This does not reduce the importance of variability in health-care spending overall, since estimates of under-65 spending show as much variability as spending on the over-65 population. In other words, if there is as much variation in the younger population as in the older population—and it appears there is—then there is as much potential for savings in the younger population as in the Medicare population.

Finally, we view the goals of restraining Medicare cost growth and of improving the quality of care for those over 65 as valuable in their own right. The growth in spending on Medicare (and Medicaid) represents the greatest threat to the future fiscal stability of the U.S. government.

The Dartmouth Atlas is often cited as a source for the estimate that 30% of the nation's spending on health care is unnecessary. What is the evidence supporting that claim?
The Dartmouth approach was to ask how much might be saved if all regions could safely reduce the volume of care to the level observed in low-spending regions that have equally good outcomes. We found estimates ranging from 20% to 30% but view these as an underestimate, given potential savings even in regions that are already low-spending.

At least three other prominent research groups have estimated that 30% of U.S. health-care spending is wasted: the New England Healthcare Institute, McKinsey and Company, and Thomson Reuters.

And at least three other prominent research groups have come to 30% waste estimates: the New England Healthcare Institute, McKinsey and Company, and Thomson Reuters.

Would it be possible to base reimbursements to regions and hospitals on the Atlas findings?
The Atlas measures of cost are reliable, but they should not be used to set payment rates until hospitals and their associated physicians can be organized in ways that allow them to improve patient care. Atlas measures do provide useful insights for regional health systems and individual providers who wish to consider how they might reduce over-treatment, improve care, and curb spending.

But we recognize the challenges of measuring costs and quality accurately. How do we adjust reliably for the fact that some hospital-physician networks treat sicker patients than others, without confronting the risk-adjustment bias that MedPAC encountered? What are the best ways to measure and reward high-quality care? These questions need to be addressed as the U.S. health-care system begins to wrestle with timetables in the new health-care reform legislation.

Still, the bottom line is that we believe there is enormous scope for improving the efficiency and quality of U.S. health care. The Dartmouth research suggests that improvements in both cost and quality can be achieved through the development of new models of payment that reward providers for improving quality, managing capacity wisely, and reducing unnecessary care.

More information on this research is available on the Atlas website. In addition, Dartmouth Medicine has often covered the findings of the Atlas researchers. Links to past articles on the subject, and to the Atlas website, are available in this article's Web Extras.

If you'd like to offer feedback about these articles, we'd welcome getting your comments at DartMed@Dartmouth.edu.

These articles may not be reproduced or reposted without permission. To inquire about permission, contact DartMed@Dartmouth.edu.

Back to Table of Contents

Dartmouth Medical SchoolDartmouth-Hitchcock Medical CenterWhite River Junction VAMCNorris Cotton Cancer CenterDartmouth College