General Background on the MA & PDP CAHPS Survey
General background information on the MA & PDP CAHPS survey can be found here.
Individual-Level Weight Construction in MA & PDP CAHPS
Individual-level weights are used for research purposes. The weights used for Star Ratings are at the plan benefit package level and are described here.
Information on construction of individual-level weights in MA & PDP CAHPS can be found here.
Nonresponse Adjustment of the Medicare CAHPS Surveys in Research Papers Using the Individual-Level Weights
The distributions of characteristics of respondents to the MA/PDP/FFS CAHPS surveys differ from the corresponding distributions in the entire CAHPS-eligible Medicare population for three reasons: (1) the sample design is not proportional, since target sample sizes are similar across contracts with very different eligible enrollments; (2) random sampling of the survey sample causes small variations in representation of the various subgroups; (3) unit (beneficiary) response has been found to follow patterns typical for health surveys (Beckett et al. 2019, Beckett et al. 2016, Burkhart et al. 2019, Elliott et al. 2019, Klein et al. 2011), including higher response rates for non-Hispanic White beneficiaries than for other racial/ethnic groups, higher response rates for older ages through age 79, and lower response rates for low-income beneficiaries.
Base weights, reflecting the probability of being sampled under the sample design, are constructed to account for the first issue listed above. Individual-level[1] calibration weights are then constructed to address all three causes of unrepresentative sample distributions. These are designed to weight means of each characteristic (and selected combinations of characteristics) among the respondents to match the corresponding means for the entire population, while perturbing the original[2] base weights as little as possible. Variables considered in this step include those that are known for the entire population before sampling. We call these individual-level weights because given the large number of possible combinations of these variables, each respondent’s weight must be provided as if unique and attached to a person-level file. These individual-level poststratification weights account for sample design and nonresponse (Deming and Stephan 1940, Purcell and Kish 1980) by matching weighted sample and enrollment populations in each Medicare contract-by-county combination on demographic characteristics, Medicaid eligibility/low-income subsidy enrollment status, enrollment in a Special Needs Plan, and zip-code level distributions of income, education, and race/ethnicity. To the extent that this fully accounts for any differences in probability of response associated with CAHPS scores, the weights allow for unbiased estimation of CAHPS measure scores.
Research papers using MA/PDP/FFS CAHPS data, including linked SEER-CAHPS data, should use individual-level weights, particularly in reporting research findings that are meant to describe national distributions, including those of cancer-relevant measures. These weights do not obviate the need in research applications for case-mix adjustment, which facilitates comparisons of contracts or groups by accounting for factors such as differences in scale use.
References:
Beckett, M. K., M. N. Elliott, Q. Burkhart, P. D. Cleary, N. Orr, J. A. Brown, S. Gaillot, K. Liu, and R. D. Hays. 2019. "The effects of survey version on patient experience scores and plan rankings." Health Serv Res 54 (5):1016-1022. doi: 10.1111/1475-6773.13172.
Beckett, Megan K., Marc N. Elliott, Sarah Gaillot, Ann Haas, Jacob W. Dembosky, Laura A. Giordano, and Julie Brown. 2016. "Establishing Limits for Supplemental Items on a Standardized National Survey." Public Opinion Quarterly 80 (4):964-976. doi: 10.1093/poq/nfw028.
Burkhart, Q., N. Orr, J. A. Brown, R. D. Hays, P. D. Cleary, M. K. Beckett, S. E. Perry, S. Gaillot, and M. N. Elliott. 2019. "Associations of Mail Survey Length and Layout With Response Rates." Med Care Res Rev:1077558719888407. doi: 10.1177/1077558719888407.
Deming, W., and F. F. Stephan. 1940. "On a Least Squares Adjustment of a Sampled Frequency Table When the Expected Marginal Totals are Known." Annals of Mathematical Statistics 11:427-444.
Elliott, M. N., D. J. Klein, P. Kallaur, J. A. Brown, R. D. Hays, N. Orr, A. M. Zaslavsky, M. K. Beckett, S. Gaillot, C. A. Edwards, and A. M. Haviland. 2019. "Using predicted Spanish preference to target bilingual mailings in a mail survey with telephone follow-up." Health Serv Res 54 (1):5-12. doi: 10.1111/1475-6773.13088.
Klein, David J., Marc N. Elliott, Amelia M. Haviland, Debra Saliba, Q. Burkhart, Carol Edwards, and Alan M. Zaslavsky. 2011. "Understanding Nonresponse to the 2007 Medicare CAHPS Survey." The Gerontologist 51 (6):843-855. doi: 10.1093/geront/gnr046.
Purcell, Noel J., and Leslie Kish. 1980. "Postcensal Estimates for Local Areas (Or Domains)." International Statistical Review / Revue Internationale de Statistique 48 (1):3-18. doi: 10.2307/1402400.
Summary of Comparison of Linear Mean Scoring with Top-Box Scoring for Medicare CAHPS Measures
Official scoring of Medicare CAHPS is linear mean scoring. A summary that compares linear mean scoring with top-box scoring for Medicare CAHPS measures can be found here. The summary includes median reliabilities for Medicare Advantage contracts calculated from 2017 CAHPS data.
Internet Citation
Please use the following citation when referencing material on this website. [www.MA-PDPCAHPS.org] Centers for Medicare & Medicaid Services, Baltimore, MD. Month, Date, Year the page was accessed. www.MA-PDPCAHPS.org.
This page was last modified on 9/6/2023