Why Nobody Believes the Numbers:
The Outcomes Measurement Guide for Grown-Ups

Sign up to receive Al's upcoming book at a discount, and get the introduction free today.

Find Out More

Complete Document Library

 

Each of the documents described below was developed as part of Consortium efforts to facilitate disease management contracting.  Almost any individual or organization with an interest in disease management will find at least some of these items helpful.  In order to facilitate document selection, each item is marked with one or more of eleven codes:

 

  Code Description
  1 Helpful in evaluating vendors for employers and health plans
  2 Supports benchmarking of internally developed or outsourced programs
  3 Of strategic interest to vendors, pharmaceutical companies, health plans, employers or investors
  4 Good data on industry
  5 Intended for contracting and/or implementation assistance
  6 Process Facilitation
  7 Helpful for payors “building” programs internally or vendors  designing them for sale
  8 Legal and Regulatory Issues  
  9 Especially for Medicaid
  10 Especially for “builders”
  11 Especially for Employers
  12 Especially for Vendors selling to non-Consortium-member Employers
  13 ROI/Savings measurement and assistance
  14 Pricing Benchmarks
  15 Peer-reviewed insights into methodology for DM
  16 Especially for wellness
  17 Especially for Predictive Modeling
  18 Especially for Medicare
  19 Especially for Medical Home

 

All document page listings are single-spaced, 10-point type, unless otherwise indicated or obvious from the context.  Page numbers are approximate and will vary according to the configuration of the software package into which they are downloaded.   Prices listed are for non-members.  Every document is free for payor members.   Credit cards are accepted.  Prices are for electronic transmission only.  Any other type of transmission has a $10 service charge/article and a $50 minimum.  The year in parentheses represents the year of authorship.  If updated, the year of the most recent update appears following the authorship year.

 


 

TITLE AND BASIC INFO

SUMMARY DESCRIPTION

“Apples to Apples” Final RFP Template (1,5,6,11), $2000, 8 pages; rfp-apples-to-apples.doc  (1999-2010)

The “special sauce” that ensures getting a “goof-proof” bid from finalist vendors, with near-absolute comparability between bids and assumptions.  No meaningful biostatistical or measurement fallacies are possible in responding to this.  Available free only as part of a brokered bid process, to full Consortium members only.   Otherwise, $2000.  Available for COPD, CHF, CAD (especially CAD), diabetes, cancer, asthma, ESRD, population management. 

 

Asthma, Codes  for Stratification (2,4,5,7,10), $500; 2 pages asthma codes.doc  (2000-2006)

 

Concise “consensus” of in/out and red/yellow/green claims-and drug-based stratification criteria compiled from various sources and reviewed by pulmonologist panel.  Much better at avoiding “false positives” and “true negatives” than any current vendor or public-domain tool based on claims and/or drugs.

 

Balancing Your Medicaid Budget (9); $500; 28 slides; Balancing Your Medicaid Budget.ppt (2002-2007)

 

Overview of all the opportunities available to Medicaid programs in all medical management areas, including dual-eligibles and TANF as well as disabled.

Best Health Plans and Vendors in Disease Management (3,4,11); $1000 non-members; $500 members; 20 pp. Best Health Plans and Vendors.pdf  (2004-2010)

The Health Industries Research Companies has made this report exclusively available to DMPC members at half price.    It selects the 30 best health plans, employers and states, and 11 best vendors in disease management and has other information on the industry, its growth prospects, and the role of pharmaceutical companies.  28 of the 30 health plans are DMPC members.  Note:  this is the only document not free with full payor membership.

 

Biggest Mistakes in Disease Management Contracting (5,13), $300; 3 pp. The Biggest Mistakes in Disease Management Contracting.doc (2001-2002)

 

A compilation of the six biggest contracting mistakes in DM, essential for any health plan or other payor to know.

Biometric monitoring:  A Report and Vendor comparison (1,3,4,5), $200; 53 pp. of  tables; biometric monitoring.doc (2001)

 

Along with an introduction, 10 companies’ biometric monitoring offerings compares side-by-side. 

Budget by Cost Component for DM Program (2,7), $500; 1 slide; dollars spent.ppt (2002 - 2008)

Summary of DM program spending per claims dollar.  Spending is broken down into 9 cost categories.  It represents an average of 3 vendors and 3 health plans for which data was received (a total of 18 average datapoints).   In addition, a full call-center staffing model for Medicaid and Medicare is typical staffing.xls

“Build” vs. “Buy” caveats (2,5) $20; 4 pp.;artchfbu.doc  (1997)

A series of case studies in CHF which highlight stumbling blocks of one plan’s “build” strategy, in order to allow other plans to anticipate the same.

 

“Build” vs. “Buy”:  The Last Word; (2.5) $300; 3 pp. + 20 slide powerpoint  DMColum1.doc, build-vs-buy.ppt  (2000-2007)

 

The decision rule to determine which to do in what circumstances, period.

Cardiac Package for “Building” programs internally (2,5,7,10) $400 non-members; $200 members.  2 Spreadsheets  Cardiac Package.xls (2002)

 

(1)     ROI analysis by cost component for internal cardiac program, both <65 and >65, easily laid out to become “transparent” for any user.

(2)     Average of several health plans’ incidences and prevalences for 6 cardiac-related events and procedures

 

“Cheat Sheet” for Vendors in Employer Negotiations (5, 11,13) $500 non-members; $200 members; 3 pages. How to Take Advantage of Benefits Consultants.doc (2004)

 

Details seven common (if not universal) mistakes in benefits consultant RFPs and contracts, and how many vendors take advantage of those mistakes to create savings metrics which ensure they will “hit their numbers.”

Claims Patterns Identifying Members  (10,13); $200; 4 pp. Extraction Algorithms.doc (2004-2006)

Official DMPC claims extraction algorithms (the patterns of claims) for CAD, CHF, diabetes, COPD.  (Asthma is a combination of extraction and stratification and is Asthmacodes.doc)

 

Conference listings for 2011; free Conference Listings  (2011)

 

Identifies particulars and contact points for all DM-oriented conferences.

Contracting with Vendors (1,5,11,13); $50; 2 pp. Contracting: 18 points.doc (2009)

The 18 most common contractual mistakes in care management contracting.

Corporate Revenue Projections for DM programs (3,4), $500, spreadsheet   vendor projections.doc (1998-2008)

 

For each independent (and some non-independent) disease and population management company, we estimated and tracked revenues for the last ten years

 

Diabetes Stratification Model  (2,4,5,7,10) $100  2 pp.  Diabetes Codes (2001-2003)

 

Very simple medical claims-based stratification tool for diabetes believed to be better than any individual vendor’s tool.

 

Disease Management Encyclopedia (1,2,3,4,5,6,7,10) $100; 42 pp. Disease Management Encyclopedia.doc (2002)

 

The single most comprehensive source of information, advice and data on disease management available anywhere.  Price has been reduced because the information is a bit dated now

Disease Management Industry Overview (1,3,4), $500, 40 slides.  DM Industry overview.ppt   (1998-2006)

Presentation format describing segment size, competitor market shares, key success factors, pharma program overviews (and why some are failing), along with descriptions of large health plan strategies

 

Disease Management Vendor Profiles (1,3,4,6,11) $500 with ratings and quality comparisons; $250 with listing only; 32 pp.vendors.doc  (1997-2010)

Names of 160 DM independent companies, together with (in most cases) contacts, phones and/or addresses and/or faxes, emails, key contracts, other profile information.  $500 version with ratings and comments is our best-seller.   Ask for the sample page, which is free.

 

Disease Prioritization Matrix, (2,3,4,6,9,10) $100; discus.doc, incidenceprevalence.xls  (1998-2010)

 

For ten different outsourced patient management opportunities, this Excel spreadsheet will tell you what your likely savings are, after you enter some “macro” costs and demographics about your plan.    The “Discus.doc” companion includes a cheat sheet and an hour of consulting to help you customize your estimates

 

 

Disease Prioritization Matrix, estimated (2,3,4,6,10,11,13) $100; 3 pp. prioritization matrix.doc (1998-2007)

 

Also available specifically for Medicaid (9)

For 12 different outsourced patient management opportunities including hospitalist, compliance and population management programs, this document lists likely size of medical losses affected (roughly adjusted for Medicare and Medicaid), expected savings, ease and timing of implementation, and non-price reasons to undertake or not undertake a particular disease category. 

 

Does Quality Matter? (1,2,3,5,7), $100, 3 pp. DMColumn7.doc   (2001-2005)

 

The shocking (ok, not that shocking, but pretty counterintuitive) truth about the role of quality in DM vendor selection and the pluses and minuses of various measurement techniques for it…

Employers: Estimating “soft” savings (11,13, 16); $100; 2 pp Soft Savings Estimator.doc (2008)

 

Best example of how-to estimate for time lost to absenteeism

Employers and Unions, A Primer on Disease Management (3,5,11), $200, 38 slides Disease Primer.ppt (2000)

 

An introduction to disease management for a lay audience with more-than-usual emphasis on the health benefits for employees and correspondingly less (though still some) on financials.

Employers and Unions, A Primer on Disease Management (3,5,11), $200, 3 Pages What Unions Should Know.doc (2000)

 

A Q&A designed to introduce unions (but can easily be customized for non-union workforces) to disease management

Employer How-To Package (3,5,11) $100, 7 pp.  EBNColumns.doc (2005-2007)

 

A collection of all the 2005 Disease Management columns for Employee Benefit News. for procurement and measurement

 

ESRD:  Disease Management for Dummies (3) $100, 2 pp. DMColumn6.doc (2000-2008)

 

A listing of the many reasons why ESRD is an easy but nonetheless lucrative program for a health plan to undertake.

Event Rate Measurement:  A Complete Package Event Rate Plausibility Suite  (1,2,3,4,5,7,12, 13,15,19)   $1000; 50 pages including slides  (2009-2010)

 

Powerpoint, plus data collection template, plus how-to guide for event-based measurement

Future Trends in Disease Management Future Trends (3) $100; 9 pp. (2009-2011)

 

Look ahead into the future of DM and social media, DM and time-to-contact innovations, DM and wellness, and many more soon-to-come innovations

 

Health Plan Use of Member Data Use of Member Data (3,8,12) $$50; 1 p. (2010)

Marketplace observations (as opposed to strictly legal viewpoint) on how health plans use data.  Essential for any first-time entrant into the care management field

 

How to Pay for DM Programs (5) $100; 3 pp. DMColumn5.doc (1999-2002)

Article describing how the population-based payment mechanisms are preferred, and how to solve the problems which population-based payment mechanisms present.  Also:  How to evaluate and negotiate  “tiered” payment systems.

 

Incidence, Prevalence and cost of all major disease categories (2,3,4,10,13); $500; spreadsheet incidenceprevalence.xls (1999-2010)

 

One of our “special sauce” documents.  Reflects the latest changes in incidence, is adjusted for age and payor category.  All cost information is Consortium-approved so there is no issue of people using different definitions.  Useful as benchmark to check your own calculations of your own population

Incidence, Prevalence and Cost of All Major Disease Categories, Group Health of Puget Sound (2,3,4,10);

$20; 1 pg. Incidence-prevalencePuget.xls  (1999)

 

The only HMO ever to analyze its entire claims database rigorously enough to publish the results.     (Note:  there are problems with the data they may not even be aware of, mentioned in the analysis.  However it is still helpful.)

 

Internet-based Disease Management Programs (1,3,5); $20, 3 pp. articleinternet.doc (2000)

 

Brilliant article on the synergy between the internet and disease management, and what to look for in disease management websites, and the role of websites in disease management programs. 

Internet-based Disease Management:  What Went Wrong (1,3,5) $20, 3 pp. articleinternetfollowup.doc (2001)

 

Article on why the above allegedly brilliant article was totally wrong and why the internet is a totally ineffective venue for disease management

Letter of Intent to Purchase DM Services (5,6); $200, 2 pp.loi.doc  (1998-2010)

An excellent example of a Letter of Intent for full-risk services (available for many different disease categories) capturing the issues which are most important to both parties and creating incentives for early implementation without creating perverse incentives.

 

Market Size Potential for DM in HMOs and other payors (3,4); $100; 2 spreadsheets marketsize.xls (2000-2010)

 

This concise but powerful analysis shows the likely “endgame” size of the market – and how it is derived – for HMOs separately, and then for the other segments combined.

Measurement of the Impact of Medical Management (2,3), $50, 10 pp.   Medical Management.doc (1999-2005)

 

A “how to” guide for measuring the impact of a health plan’s medical management program in total and by components, including disease management and population management.  Note:  There is no clear “answer” so if you are seeking one, you will be less than satisfied.

Medicaid:  Lessons Learned (9.15); $300 3 pp. Lessons Learned from failed Medicaid RFPs.doc  (2006)

 

An engineer learns more from one bridge which falls down than from 100 which stay up.   Several are excellent vehicles for learning what mistakes to avoid in the RFP.  This document provides examples

Medicaid Disease Management Savings Opportunities (9, 15), $300, 15 pp. Savings in Medicaid.doc (2003)

 

The seminal article on Medicaid disease management contracting, appeared in Disease Management Journal in 2004

The Medicaid Report (9); $2000 non-member, $500 member, free for state members; 32 pp.  Medicaid Opportunity Report (2002-2007)

The single most comprehensive and detailed listing of all opportunities available to state Medicaid programs in all aspects of medical management in which voluntary, guaranteed-savings programs can be implemented.  Includes a spreadsheet so that states may calculate their own savings, which should be 3-4% of the entire Medicaid budget.

 

Medicaid:  A Primer (3); $200; 30 slides Medicaid Primer (2003)

The terms, the rules, the economics, the market size, the nature of and issues with disease management…everything you need to know to talk intelligently with a state about disease management

 

Medicaid Disease Management:  What States and Pharmaceutical Companies Need to Know (3,4,5,9), $100; 14 pp. Article on Medicaid (2002)

 

No state agency or oversight authority or pharmaceutical government affairs department should be without this.

Medical Home:  Debunking the Myth of Savings (9,13,19) $2000 (non-member), $500 (member); spreadsheet plus powerpoint plus backup data Medical Home Package (2009)

Turns out (surprise) Mercer did the analysis wrong for North Carolina Medicaid.  Instead of saving $300-million/year, the model costs $400-million/year.  This package shows the whole analysis.

Medicare Reimbursement:  How to Select a Vendor  (18) $50; 2 pp. Questions a Health Plan Should Ask.doc (2006-2007)

 

This covers the “greatest hits” of questions which an HCC coding vendor should be asked…together with the best-practice answers.  Using this will facilitate your consulting-firm selection process

Medicare Reimbursement:  Revenue Maximization Strategy (18); $100; 2 pp. HCC Coding Strategy.doc (2007)

There are some little-appreciated strategies which can raise your HCC score by 10 basis points over and above any other strategy.

Medicare Stars Program Suite Medicare Stars  (13,18) $200;  9 Documents plus spreadsheet (2010)

 

Collection of hard-to-find public domain materials together with easily used spreadsheet facilitating the value analysis of Stars

Outcomes Measurement for Dummies…and Smarties, Volume 1 (1,2,3,4,5,7,12, 13,15,19) Outcomes Measurement Volume 1.doc  $495 non-members, $150 members; 34 pages (2010)

Comprehensive guide to plausibility analysis using real-life examples.  Also one of two pre-reading preparation tools for Critical Outcomes Report Analysis

 

Outcomes Measurement for Dummies…and Smarties, Volume 2 (1,2,3,4,5,7,12, 13,15,19)   Outcomes Measurement Volume 2.doc  $495 non-members,  $150 members; 32 pages (2011)

 

Mathematical proof of invalidity of “consensus guidelines,” indispensable for valid outcomes analysis.

 

Outcomes Measurement for Dummies…and Smarties, Presentation version (1,2,3,4,5,7,12, 13,15,19)   $500;  80 pages Valid and Transparent Outcomes.ppt (2010)

 

Powerpoint version of above.

Pharmaceutical DM Programs:  A listing (1,3,4);  $20, 10 pp. of tables Pharmaceutical Disease Management Initiatives.Doc   (1999-2001)

 

A tabular comparison of the 14 best-known pharma-sponsored DM divisions or subsidiaries (excludes simple “programs” which are done by the marketing department), including comments, sample accounts, program components.

Pharmaceutical DM Strategy White Paper (3), $100 (member), $200 (non-member), 24 pp. Pharmaceutical DM Strategy White Paper.doc (2001-2004)

The “answer” to how a pharmaceutical company should strategically view disease management, period.  This document carries a considerable extra cost, which also covers a day of on-site consulting to elaborate on it.  This is also fully guaranteed in advance to be the right answer, in your opinion, or you can keep the White Paper and still receive a full refund.

 

Physicians, Full-Risk (3,5); $100;  3 pp.    dmcolum3.doc (1999-2003)

Tip sheet on how to “sell” disease management to physicians who are at full risk already.

 

Predictive Modeling Effectiveness (17); $50; 2 slides Predictive Modeling Effectiveness.ppt (2006)

 

Results of a careful analysis of five (unnamed) predictive modeling packages in their ability to identify the following year’s high-cost members and also members transitioning from low-cost to high-cost.

Pricing Strategy (14); $200 2 pp. pricing strategy.doc (2006)

For buyers and sellers and re-sellers to self-insured employers.  A concise but insightful look at what works and does not work in pricing strategies.

 

Pricing by Disease (14) $300 2 pp. pricing by individual disease.doc (2004-2010)

PMPM pricing by disease for the 5 key diseases, including discounts for multiple diseases – 0% and 100% risk for commercial and Medicare.

Pricing and Scoring of RFPs (14); $1000—non-members $200--members; spreadsheet abstract—blinded.xls (2004-2009)

 

PMPM 5-disease, by year and level of risk, across a range of (unnamed) vendors.  Includes ROIs and effect of guarantees on pricing, savings and ROIs.  Demonstrates how scoring works

 

Pricing and Scoring of RFPs—Medicaid (9,14) $300—spreadsheet  Medicaid abstract.xls  (2006)

Scored PMPM for Medicaid disabled bid.  Demonstrates Medicaid scoring of savings , fees, and ROI.

Process Steps in Outsourcing (6); $500; 5 pp.  Process Steps.doc  (1998-2004)

 

Detailed description of each of the thirty process steps involved in selecting a disease and developing an outsource program for it.

 

Proposals:  Reading and Writing (5)  $100; 7 pp.   Howtoreadaproposal.doc (1998)

Annotated model proposal for Member and vendor instruction and enlightenment.   One of the bestsellers.  Based on ESRD example but could be used for any disease.  

 

RFP Response, Unabridged Sample Completed RFP.pdf  (2,4,12,14) $500; 100+ pages (2010)

Sample complete response (including pricing) to comprehensive RFP

 

Quality and Disease Management (1,2,7,10, 15); $100, 5 pp. Article on Quality in Disease Management.doc (2003-2005)

The straight scoop on quality measures—validity, relevance, and correlation with actual outcomes.  The only guidance you’ll ever need on how to develop a quality measurement strategy.

Request for Information for Disease Management Procurement:  All the Questions You Can Think Of (5)  $200; 20 pp., RFI.pdf (2002-2009)

You can pick-and-choose RFI questions from this very comprehensive list and then add your own, simplifying your own process dramatically.

Reporting Template:  What you Should Demand from a Vendor or Produce as an Internal Program (1,2,5,7,10) $200; 17 slides Template for Reporting.ppt (2003)

Standard set of deliverables which any program should produce for sponsor.  You can use this to negotiate how much less is produced for your own ASO employer customers of various sizes

Return on Investment:  Myth and Fact (3,13, 15) $50; 3 pp. Return on Investment.doc  (1999-2003)

 

Ten myths about the financial side of DM exploded.  Very helpful for internally selling the concept of DM outsourcing, as well as bringing people in one’s own department “up to speed”.

 

Small Group Measurement (2,7,13,16) $100; 5 pages small group measurement.doc (2007)

 

The DMPC Outlines Guidelines are silent on “Small” groups, which we define as <50,000 lives.  This guide fills that niche, describing six ways to measure.  Payors adopting at least two of those measurement techniques may want to apply for Small Group Measurement Certification

Staffing Summary, detailed (2,4,5,7,10), $500; typical staffing summary, blinded.xls (2007)

 

Average of several places, 15 job titles, workload, salary range, comments; Medicaid, Medicare, Commercial.   Very detailed.

Timeline and GANTT Chart for DM Vendor Selection and Contracting (5); $100; 1 spreadsheet timeline.xls (2002-2010)

 

Developed from dozens of projects to ensure optimal implementation and to start all concurrent activities in a timely way, to prevent unnecessary delays.

Vendor Ratings (1,3,4); $500, 28 pages of tables vendor ratings.doc  (1999-2009)

A must-have for any health plan, venture capitalist, or vendor, period.  Offered in conjunction with the vendor listings

 

Wellness Savings Calculator  (1,5,13,16); $200, spreadsheet  wellness calculator.xls (2009)

Foolproof way to tell whether your wellness program is likely to save money and if so, how much?

 

 


The following published and peer-reviewed articles are guest-authored by Ariel Linden DrPH, the leading DM  measurement researcher in the United States.  Each is available for $100 for DMPC members and $200 for non-members  They would all be “15” on the number key.

 

TITLE AND BASIC INFO

SUMMARY DESCRIPTION

Linden A, Adams J. Determining if disease management saves money: an introduction to meta-analysis. J Eval Clin Pract. In Press

An introduction to the meta-analytic technique as a means of evaluating a DM program’s overall effectiveness across various populations, diseases, payors, etc. 

Linden A, Trochim WMK, Adams J. Evaluating program effectiveness using the regression point displacement design. Eval Health Prof. In Press

An introduction to a novel evaluation design originally developed by William Trochim and Donald Campbell. In this paper, the technique is applied to various healthcare settings, in particular, where pilot programs are initiated before widespread adoption.

Linden A. What will it take for disease management to demonstrate a return on investment? New perspectives on an old theme. Am J Manage Care 2006;12(4):61-67.

A benchmark paper in which it is demonstrated that DM economic effectiveness should be measured via utilization and not cost. Additionally, a need-to-decrease (NND) analysis is introduced to assess whether there is sufficient opportunity for a DM program to have an impact on the population.

Linden A, Butterworth SW, Roberts N. Disease management interventions II: What else is in the black box? Dis Manage. 2006;9(2):73-85.

A follow-up paper to the earlier published DM interventions paper. Several new behavioral change models are introduced, including motivational interviewing, which is the newest technique used for health coaching.

Linden A. Evaluating the effectiveness of home health as a disease management strategy. Home Health Care Manage & Pract. 2006;18(3):216-222.

This paper raises the methodological issues facing home health programs in showing their value in disease management. It then offers several evaluation designs suited for this unique setting.

Linden A, Adams J, Roberts N. Evaluating disease management program effectiveness: an introduction to the regression-discontinuity design. J Eval Clin Pract. 2006;12(2):124-131.

This paper introduces the regression discontinuity technique, possibly the most robust observational study design that can be applied to DM program evaluations.

Linden A. Measuring diagnostic and predictive accuracy in disease management: an introduction to receiver operating characteristic (ROC) analysis. J Eval Clin Pract. 2006;12(2):132-139.

This paper introduces readers to the ROC analysis which is uniquely suited for assessing the sensitivity and specificity of predictive models and algorithms used for identifying suitable program participants.

Linden A, Adams J, Roberts N. Strengthening the case for disease management effectiveness: unhiding the hidden bias. J Eval Clin Pract. 2006;12(2):140-147.

Observational study designs are always subject to bias. This paper introduces a method for assessing whether results of a DM program evaluation are robust enough to overcome the influence of bias.

Linden A, Adams J. Evaluating disease management program effectiveness: an introduction to instrumental variables. J Eval Clin Pract. 2006;12(2):148-154.

Instrumental variables are typically used in econometric studies to control for bias. In this paper, the concept is applied to actual DM program data.

Linden A, Roberts N. Using visual displays as a tool to demonstrate disease management program effectiveness. Dis Manage. 2005;8(5):301-310.

Evaluating program effectiveness is one thing, but explaining them to stakeholders is another. This paper provides several different visual displays that will help explain the complex data used in DM.

Linden A, Roberts N. A users guide to the disease management literature: recommendations for reporting and assessing program outcomes. Am J Manage Care.2005;11(2):81-90.

Both peer-reviewed manuscripts and press releases may lack in accurate information for readers to assess their value. This paper introduces the specific criteria that should be followed by authors when writing a manuscript, and by readers when evaluating the value of the paper or press release in question.

Linden A, Adams J, Roberts N. Evaluating disease management program effectiveness: An introduction to the bootstrap technique. Dis Manage and Healt Outc. 2005;13(3):159-167.

Typically DM programs are evaluated using standard parametric statistics, such as t-tests, least-squares regression, etc., which are based on averages and standard deviations. However, much of the data used in DM is based on counts or rates, and have large variances in the data. The bootstrap technique is a non-parametric alternative that allows the evaluator to use additional measurement types that are more suitable to the data. This paper explains the technique and provides several examples using actual DM data.

Linden A, Adams J, Roberts N. Using propensity scores to construct comparable control groups for disease management program evaluation. Dis Manage and Healt Outc. 2005;13(2):107-127.

The propensity scoring technique is rapidly becoming a mainstay evaluation tool in DM. It is based on the concept of matching participants receiving the DM intervention with non-participating controls. The propensity score is a composite score of many baseline characteristics. This method, when used in conjunction with a sensitivity analysis (see Linden A, Adams J, Roberts N. Strengthening the case for disease management effectiveness: unhiding the hidden bias. J Eval Clin Pract. 2006;12(2):140-147), may be as robust as the randomized controlled trial.

Linden A, Adams J, Roberts N. Evaluating disease management program effectiveness adjusting for enrollment (tenure) and seasonality. Res Healthc Fin Manage. 2004;9(1):57-68.

Two important confounding variables that should be adjusted for in any DM evaluation are an individual’s length of time in program (tenure) and the calendar month (due to seasonality). This paper introduces a model that adjusts for these two variables and demonstrates how it can be interpreted.

Linden A, Roberts N. Disease management interventions: What’s in the black box? Dis Manage. 2004;7(4):275-291.

This paper introduces several basic psycho-social behavioral models that should be considered for implementing in a DM program intervention.

Linden A, Adams J, Roberts N. Evaluating disease management program effectiveness: An introduction to survival analysis. Dis Manage. 2004;7(3):180-190.

Survival analysis (also called time-to-event analysis), provides a unique method for evaluating DM program effectiveness. All participants contribute to the model, regardless of their length of time in the program. These are called censured cases and would typically not be included in standard designs if their tenure was less than 6 month or a year. This paper introduces the method and presents an actual analysis with its interpretation.

Linden A, Adams J, Roberts N. The generalizability of disease management program results: getting from here to there. Manage Care Interface. 2004;17(7):38-45.

Most studies in DM are concerned with internal validity, primarily the impact of selection bias and regression to the mean. However, external validity is equally as important. This paper introduces methods to maximize the generalizability of DM program outcomes.

Linden A, Adams J, Roberts N. Using an empirical method for establishing clinical outcome targets in disease management programs. Dis Manage. 2004;7(2):93-101.

One of the biggest problems facing DM is in knowing how to set reasonable targets for clinical outcomes. This paper provides a simple empirical method for deriving those targets.

Linden A, Adams J, Roberts N. Evaluation methods in disease management: determining program effectiveness. Position Paper for the Disease Management Association of America (DMAA). October 2003.

This paper was written at the request of the DMAA in 2003. It is intended to provide an overview of the bias inherent in the currently used total population approach and propose alternative methodologies for evaluating DM program effectiveness.

Linden A, Adams J, Roberts N. Evaluating disease management program effectiveness: An introduction to time series analysis. Dis Manage. 2003;6(4):243-255.

Time series analysis (TSA) should be used as a standard evaluation technique in DM. This paper provides a comprehensive discussion of TSA and provides direction of how it should be used and results interpreted.

Linden A, Adams J, Roberts N. An assessment of the total population approach for evaluating disease management program effectiveness. Dis Manage. 2003;6(2): 93-102.

This paper presents a comprehensive critique of the most common methodology used for evaluating DM program effectiveness.

Linden A, Roberts N, Keck K. The complete “how to” guide for selecting a disease management vendor. Dis Manage. 2003;6(1):21-26.

As the title states, this is a complete guide for selecting a DM vendor.


Disease Management Purchasing Consortium International, Inc. .

890 Winter Street, Suite 208
Waltham, MA 02451
Phone: 781 856 3962
Fax: 781 884 4150
Email: alewis@dismgmt.com