Overview

Emerging research on administrative burden has highlighted the need for survey measures that capture people’s experience of government as onerous. This study presents a measure of experienced administrative burden, based on a survey of social welfare users. Using psychometric scale development techniques and a split sample approach, we developed both a single-item and three-item scale that can be applied in user surveys for both research and practitioner purposes. We used predictive validity tests to assess both measures, finding that people are more likely to report burden when they have poorer health, lower education, experience short-term financial scarcity, are younger and have less program-specific experience, as anticipated.

Principles for Measuring Burdens

Measures should reflect experiences

Questions about burdens should be anchored in specific, recent experiences to enhance recall and minimize bias.

Capture multiple dimensions of burdens

Items should encompass learning, compliance, and psychological costs. Learning costs incorporate both the search for relevant information and understanding the content of the information. For compliance costs, the values of effort and time are frequently evoked. Psychological costs include a sense of shame or stigma, a sense of loss of personal autonomy, stress or frustration with the process and associated outcomes.

Measures should be adaptable to different contexts and experiences

Given researchers need to specify which aspect of the policy implementation process or specific interaction they are asking respondents about, these measures are designed to be adapted to diverse citizen-state interactions.

Measuring burden should not be burdensome

We developed a concise scale so as not to overwhelm respondents and achieve higher response rates.

Survey Items

The suggested survey items were tested with actual users of a major social program, the U.S. Supplemental Nutrition Assistance Program (SNAP), but can be adapted for other programs with minimal effort. Because of the multi-dimensional nature of psychological costs, we tested a variety of items, including feelings of frustration, being in control, stress and a sense of being respected.

Below is our recommended single-item scale.

Below is our recommended three-item scale.

Validation Methods

We completed a set of predictive validity tests to examine how well the single-item and three-item scales are correlated using a simple linear regression model where experiences of administrative burdens are predicted by health, self-reported financial scarcity, age, current gender, education and years of SNAP. As anticipated, we found that poorer health, higher levels of perceived financial scarcity, and lower levels of education are correlated with higher levels of burden. We also found that people who have used SNAP for longer periods report reduced experiences of burden.

These findings offer broad support for a three-items scale that captures aspects of learning, compliance, and psychological costs, but also shows that the single overall item works almost as well, and may be easier to use in surveys where space is a consideration.

In a final series of tests, we also compared both measures of administrative burden to the federal customer experience scale (or federal CX scale). We found that the CX and burden scales capture different phenomena. The federal CX measure captures people’s broader customer experience including aspects that are correlated with different burden costs (like ease of completing an application), while the burden scale addresses those costs directly.

Funders: Schmidt Futures