This is the second in a blog series following the Center for Member Well-Being’s working group on measuring member financial well-being (FWB).
Led by Filene Fellow Dr. Mat Despard, the Financial Well-Being Measurement Working Group brings together credit union leaders to explore how to better understand members’ financial lives, measure the impact of financial well-being initiatives, and turn data into meaningful action. Want to learn more about the working group and follow its progress? Read about the full journey here.
A dive into surveys and measures of subjective financial well-being
In the kickoff session, we identified why the participating credit unions want to measure member financial well-being and the common challenges underscoring how difficult it remains to turn available data into clear, actionable insights. Two weeks later we met again, building directly on that foundation we started to explore one piece of that puzzle: the role of surveys.
In this second session, participating credit unions spent 90 minutes exploring
- Where surveys add value
- Where they fall short
- What different survey approaches participating credit unions are taking
- What credit unions need to make survey data more useful
Where do surveys add value?
Throughout the conversation, there was consensus on the premise: Credit unions cannot fully understand member financial well-being from balances, credit scores, and transaction data alone. Those indicators show what is happening in an account, but they don’t tell us about how the member is experiencing their financial life.
Those indicators show what is happening in an account, but they don’t tell us about how the member is experiencing their financial life.
The discussion, thus, gravitated toward what surveys reveal about members:
- Financial confidence
- Financial self-efficacy
- Stress or peace of mind
- Perceived debt manageability
- Goals and challenges
- Knowledge, skills, and intended behavior
Participants repeatedly returned to these concepts as information that credit unions cannot easily gather from objective data sources alone. To underscore the importance of capturing the subjective aspect of well-being through surveys, one participant connected this idea to broader affordability concerns, noting that external economic conditions and household realities may influence financial well-being in ways that do not immediately appear in account balances or credit scores.
Dr. Mat Despard also discussed the value of using questions drawn from standardized surveys to assess member experiences against broader populations. Some examples are the National Financial Capability Survey, the Financial Health Score, the Survey of Household Economics and Decision-making, and the CFPB Financial Well-Being Scale. By using validated questions, credit unions can identify where their membership scores lower than national averages(e.g., on emergency savings or financial stress), to inform product and services strategies. Standardized surveys also reduce the burden on the credit union; these surveys have been rigorously validated and tested, giving the credit union confidence that the questions will yield helpful answers.
The Challenges Everyone Agreed On
While participants saw clear value in surveys, there was little disagreement about their limitations. The main challenges identified by the group were:
Low and unrepresentative response rates
Low response rates was the most cited frustration, with several credit unions describing single-digit participation despite considerable effort. The deeper problem is who responds. Participants observed that respondents tend to be more financially stable and more engaged than membership overall, which means the members a credit union needs to hear from most are the least likely to answer. A larger membership does not solve this, it just produces more responses from the same kind of member.
Survey fatigue
Survey fatigue surfaced repeatedly throughout the conversation with some describing an environment in which members are routinely asked to complete surveys from many organizations, making it increasingly difficult to gain attention and participation. One participant observed that even substantially shorter surveys did not appear to attract meaningfully different audiences than longer versions, reinforcing that the same members who are willing to complete a lengthy questionnaire are often the same members willing to complete a shorter one.
Self-reporting and intention bias
The discussion also highlighted another challenge: people’s intentions and reported behaviors do not always align with their actual financial realities. Participants discussed how survey respondents may overestimate future behaviors or provide answers they believe reflect good financial habits. As a result, some behaviors and financial outcomes may be measured more accurately through observed financial behaviors rather than self-reporting through surveys alone.
Difficulty measuring change over time
Added to the difficulty of even getting responses, participants noted survey programs tend to break down around follow-up. Response rates fall off when members are surveyed again at three or six months, which makes it difficult to know whether financial knowledge, confidence, or behavior changed at all and whether the financial well-being program worked. That matters because knowledge and behavior decay differently: gains in knowledge often persist while changes to behavior erode. A short follow-up window can make an intervention look more effective than it is.
What participating credit unions are trying
Earlier in the session, Mat had distinguished two purposes that credit unions typically have for surveys, which come down to two questions: what are our members experiencing, and are our efforts helping? We heard from participating credit unions details of survey-based measurements they are implementing.
- Broad membership surveys to assess changes in financial well-being over time. Using the Gallup-Callahan financial well-being methodology, a participating credit union is hearing its members describe their financial realities in their own words. The survey surfaces the concerns, challenges, and stressors that traditional metrics miss, and classifies members across the Thriving, Struggling, and Suffering categories. The team’s current focus is not the instrument itself but its distribution: moving results out of periodic communications and into internal dashboards, so staff can respond to emerging needs rather than read about them a quarter later.
- Curating survey questions to evaluate program outcome. Another credit union shared a survey to collect feedback across community education, financial counseling, and wealth-building activities: what members attended, why they came, whether their confidence changed, what they plan to do next, and what they may need later. The appeal was one consistent instrument across several channels. The obstacles have been operational. Some processes require staff to send the survey manually, community partners do not reliably generate responses, and certain activities raise compliance considerations. The survey is also anonymous, which protects confidentiality but makes it impossible to connect a response to anything else the credit union knows about that member.
- Reducing friction with a checkup inside digital banking. Several participants described using SavvyMoney’s wellness checkup, which pairs self-reported responses with credit and financial data to give members a snapshot while generating data for the institution. Enrollment approaches range from opt-in to automatic, but even credit unions with substantial digital banking enrollment report that relatively few members complete the checkup, and those who do may skew more financially secure than those who do not. Several are still working out how to export, warehouse, and share the resulting data. Some are considering moving the checkup into financial coaching intake.
That last idea, putting the questions at the moment a member has already asked for help, is where the conversation got most interesting.
Rather than surveying an entire membership and hoping for participation, several participants described asking questions at the points where members have already chosen to engage. One participant described a coaching pilot that builds Financial Health Network-style questions into the intake form. The questions give coaches a picture of a member’s goals and challenges before the first conversation, establish a baseline for measuring change, and prompt members to reflect on their own situation. This contrast between broad membership surveys and intake or program-based measurement highlights an important distinction in how survey measurement can be used: each approach offers value, but each answers a different question and comes with different tradeoffs.
| Broad membership survey | Intake and program-based measurement | In-context survey questions | |
|---|---|---|---|
| Best suited for | Taking a pulse of the membership, tracking perceptions over time, benchmarking against national data | Understanding an individual member’s goals and challenges, establishing a baseline, evaluating whether a program worked | Capturing how a member feels close to the moment it matters, while they are already in the app or just after an interaction |
| Why members respond | Little immediate reason to participate | They have already asked for help or chosen to take part | The question costs seconds and appears where they already are, though they did not come looking to answer it |
| Main limitation | Participation bias, and findings that may not connect to a specific action | Reaches only members already engaged, so it says little about the rest of the membership | Completion stays low even with high digital banking enrollment, and short answers rarely explain the why |
What’s Next
Surveys are both indispensable and insufficient. While surveys are the only practical way to capture confidence, stress, and the barriers members perceive, low participation, response bias, and implementation friction stand in the way of relying on surveys alone to measure member financial well-being. This reinforced a conclusion from session 1: Credit unions need survey data to work alongside member account and product data, creating a model that can assess members’ subjective and objective financial well-being on an ongoing basis.
The session closed on a caution that applies no matter which data source a credit union leans on: Measurement tells a credit union where a challenge may exist, but a theory of change is what turns that signal into a decision. Identifying a problem does not identify the solution, and a member with non-existent emergency savings may be facing constraints that no amount of redistributed budgeting content will change.
That framing shaped how the group described what would genuinely help the industry:
- Not a new survey, but guidance on which existing instruments and questions fit which credit union needs
- Practical insight into well-fielded surveys, including timing, frequency, length, placement, and triggering event
- A way to link types of financial challenge to evidence-backed interventions, built on the questions between a measurement and a response (e.g., why is this problem occurring, why would one response work better than another, what evidence supports it, and what outcome would tell us it worked)
Session 3 takes up the other half of the equation: how credit unions are using transaction, credit, and other system-generated data to understand member financial well-being and outcomes.
