Frequently Asked Questions - Contracts
Are all MA and PDP contracts required to participate in the MA & PDP CAHPS Survey?
Medicare Advantage Organizations, 1876 Cost Contracts, Medicare-Medicaid Plans, and Part D sponsors with at least 600 enrollees as of July of the previous year are required to participate in the MA & PDP CAHPS Survey. The survey is administered to a sample of beneficiaries drawn from contract enrollees in January of the current year. While the specific dates of data collection will vary from year to year, the survey is usually administered during March to June.
Who pays for survey administration?
MA and PDP contracts contract directly with approved MA & PDP CAHPS Survey vendors to conduct the survey. Contracts communicate their choice of vendor to CMS via a web-based Vendor Authorization and Oversample request tool.
How is my contract’s eligible sample determined?
Each contract’s sample is drawn from January enrollees with continuous enrollment in the contract from July to January. Continuous enrollment in the contract is determined using CMS monthly enrollment data. If a contract member has any gaps in the CMS monthly enrollment data, he or she is excluded from the sample. Continuous enrollment is one of several eligibility criteria. CMS excludes enrollees who are under age 18, known to be deceased, known to reside in institutional settings, or reside outside of the U.S., Puerto Rico or U.S. Virgin Islands. Additionally, CMS sample procedures do not allow for the selection of more than one enrollee per household.
Where can a contract get information about Vendor Authorization?
Each November, CMS provides Medicare Compliance Officers with information about vendor authorization for the upcoming administration of MA & PDP CAHPS. Your contract’s Compliance Officer should contact firstname.lastname@example.org or 1-866-690-1650 with any questions about vendor authorization for the upcoming MA & PDP CAHPS Survey cycle.
My contract would like to request an oversample, how do I do that?
For information about how to request an oversample, your contract’s Medicare Compliance Officer should contact email@example.com or 1-866-690-1650. Oversample requests are usually due in early December for the upcoming administration of MA & PDP CAHPS.
Where can I find the annual timeline for the MA & PDP CAHPS survey cycle?
While specific dates will vary from year to year, the general timeline for the survey is:
HPMS memo publicizing key dates for contract actions is distributed in November
Medicare Compliance Officers for contracts required to report CAHPS receive instructions for vendor authorization and oversample requests in November
Contracts must authorize a vendor and request any oversample by early December
CMS begins sample selection in January, based on each contract’s current enrollment
Survey administration occurs from approximately March to June
Plan preview reports are emailed to Medicare Compliance Officers in August
Medicare Compliance Officers are mailed Detailed Plan Reports in late fall
What languages are available for MA & PDP CAHPS Survey administration?
The MA & PDP CAHPS Survey is available in English, Spanish, Chinese and Vietnamese. All contracts are required to administer the survey in English and Spanish, and contracts have the option to administer the survey in Chinese and/or Vietnamese depending on the needs of their enrollees. The Chinese translation of the survey is appropriate for Cantonese and Mandarin speakers.
Are supplemental questions allowed for MA & PDP CAHPS Survey administration? If so, are there any restrictions on the questions?
CMS allows contracts to add a maximum of 12 supplemental questions to the MA & PDP CAHPS Survey. Contracts submit supplemental items to CMS for review and approval for use via their authorized survey vendors.
Detailed guidance on the submission process and examples of questions that do not meet CMS guidelines for approved supplemental items can be found in Appendix O of the Quality Assurance Protocols & Technical Specifications (QAP&TS) manual.
As a resource for possible supplemental questions, CMS suggests the use of the CAHPS Supplemental Items for the Adult Health Plan Questionnaires posted on the AHRQ Web site. These items have been tested; however, please note that some of the items may not meet the protocols for MA & PDP CAHPS Survey supplemental items.
Are there any restrictions or limitations on using Medicare CAHPS to survey Medicare beneficiaries at other time points in the year outside the range of the official Medicare CAHPS data collection cycle?
CMS strongly encourages contracts and survey vendors NOT to ask Medicare beneficiaries any MA & PDP CAHPS Survey questions four weeks prior to, during, and four weeks after the MA & PDP CAHPS Survey administration period (that is, from February 1 to June 30 each year). Other CMS surveys are exempt from this guidance. CMS provides this recommendation to promote response rates to both MA & PDP CAHPS and additional data collection efforts.
Who will submit a Medicare Advantage contract’s CAHPS data to NCQA?
At the end of the annual MA & PDP CAHPS survey cycle CMS will submit the MA CAHPS survey data files to NCQA with calculated scores for accreditation. Note that NCQA uses a different scoring methodology from CMS in calculating results.
May contracts receive de-identified member-level response files from their CAHPS survey for quality improvement?
No, the CMS Data Use Agreement for MA & PDP CAHPS prohibits vendors from sharing beneficiary-level data. These data could be used to identify an individual (either directly or indirectly), which would violate the guarantee of confidentiality that CMS provides all survey respondents.
When are MA & PDP CAHPS Survey Reports available each year?
The MA & PDP CAHPS Survey Plan Preview Reports are generally delivered to a contract’s Medicare Compliance Officer in August of each year by email. The MA & PDP CAHPS Survey Detailed Plan Reports are generally delivered to the contract’s Medicare Compliance Officer in late fall of each year by mail. Contracts are encouraged to keep their Medicare Compliance Officer name and contact information up-to-date within HPMS.
How might CMS analysis of the survey data differ from the survey vendor’s analysis?
CMS-calculated results are official results. Every year some plans inquire about discrepancies between vendor and CMS results. Historically, discrepancies are due to factors such as vendor misapplication of forward-cleaning rules, vendor top-box scoring rather than linear mean scoring, and vendor errors in the determination of eligible surveys.
How are survey data edited and cleaned during data analysis by CMS?
A forward-cleaning approach is used for editing and cleaning survey data. This approach uses responses to the “screener” (or gate) items to control how subsequent items within the questionnaire are treated analytically, such as setting responses to a missing value or retaining the original response. Under this forward-cleaning approach, unanswered screener items are not updated or back-filled based on responses to subsequent items. The conventions and guidelines used to forward clean MA & PDP CAHPS data can be found in Chapter IX: Data Analysis and Public Reporting of the QAP&TS.
How is reliability calculated for measures?
Reliability is calculated for each contract’s score for each measure. This indicates how well the score for a single contract distinguishes its performance from that of other contracts, and it is scaled from 0 (indicating no information) to 1 (indicating perfect accuracy in comparing or ranking contracts). Interunit reliability is calculated using the formula IUR = 1−V/(V+t2), where V is the variance of the estimate of the measure for that contract, and t2 is the between-contract model variance of the means (estimated from a linear random-effects model). IUR as defined here is related to Spearman-Brown reliability but our formulation allows for variation in reliability across contracts due the varying rates and patterns of response.
Reliability of the estimated scores is affected by a number of factors, including:
• The number of a contract’s respondents who answer an item (or each item of a composite measure);
• The variability of responses within the contract;
• The amount by which contracts differ from each other nationally on that measure.
The reliability of a composite measure cannot be obtained by averaging the reliabilities of its constituent items. Measurement error decreases with multiple measurements (multiple items from the same beneficiary in each composite), so the measurement error for a composite averaging multiple items tends to be less than the average measurement error for individual items. The total number of respondents providing information toward the composite might be greater than the number responding to any one item. Consequently, a composite often has higher reliability than most or sometimes any of the items that comprise it.
Low reliability scores are defined as those with at least 11 respondents and reliability ≥0.60 but <0.75 and also in the lowest 12% of contracts ordered by reliability. These scores are publically reported and may affect Star Ratings. If reliability is very low (<0.60), the contract does not receive a Star Rating for that measure.
If two survey measures have low reliability in a composite based on the mean of scores of three survey measures, is the composite based only on one measure?
Reliabilities of individual survey measures or items are for internal QA use only; item-level reliabilities do not have implications for scoring and items are not removed from a composite because of low reliability. Only reliability of the composite has implications for scoring or reporting.
Where can I find more information on how the survey results are calculated?
Information regarding scoring and analysis of MA & PDP CAHPS Survey results can be found in Chapter IX: Data Analysis and Public Reporting of the QAP&TS.
Where can I find response rate information for the MA & PDP CAHPS survey?
CMS provides historic response rates on the MA & PDP CAHPS Survey website. The available information includes historic MA & PDP CAHPS response rates overall, by survey type, and by survey mode. Historic overall survey response rates for Medicare Advantage, Fee-for-Service, and Prescription Drug Plan coverage are also available. CMS does not make contract-level response rates publicly available.
If a contract decides to field the MA & PDP CAHPS Survey on an optional basis, will the results be used toward Star Ratings?
All contracts with fewer than 600 eligible enrollees are not required to field the survey; if the number of eligible enrollees is between 450 and 599, a contract may field the survey on an optional basis. Contracts that choose to participate will have their scores reported and used in Star Ratings.
Who can I contact with questions about my CAHPS scores and MA & PDP CAHPS Star Ratings?
Questions regarding your MA & PDP CAHPS scores should be directed to MP-CAHPS@cms.hhs.gov.
Questions regarding Part C and D Star Ratings calculations should be directed to PartCandDStarRatings@cms.hhs.gov.
Last modified on 8/2/2018