Choosing a Proxy for Change That Doesn't Betray the Long Game

You are running a job training program. You require a metric. Quickly. So you count 'number of graduates.' Easy. But what if graduates can't find work? Or worse, your staff starts enrolling anyone just to boost numbers? That is the proxy trap. A good proxy measures revision without inviting manipulation. A bad one betrays the whole mission.

Not always true here.

Most crews miss this.

Fix this part opening.

This article is for impact managers, evaluators, and funders who need to pick proxies that don't rot over time. We will walk through the hard choices, the hidden assumptions, and the practical steps to keep your long game intact. No fake stats. No easy answers. Just an honest look at what works and what falls apart.

That is the catch.

Fix this part opening.

Not always true here.

Who Needs This and What Goes off Without It

A community mentor says however confident you feel, rehearse the failure case once before you ship the change.

The impact manager who trusted graduation rates

She had everything—spreadsheets, five-year trendlines, a board that nodded approvingly. The program was getting students through high school at record speeds. Graduation rates jumped 12% in two years. Then the follow-up survey hit: 80% of those graduates couldn't hold a job interview, couldn't compute a tip, couldn't name one adult they trusted. The metric looked good; the real outcome was a lie. The impact manager lost her budget—not because the program failed, but because the proxy failed her opening. That's the quiet trap. The number goes up, the story sounds right, and nobody checks what the number actually leaves behind.

Skip that stage once.

Some proxies are worse than dishonest—they're contagious. A donor hears 'graduation rate up' and doubles down. Other funders follow. Soon the entire ecosystem optimizes for the proxy: push students through, skip life-skills coaching, ignore trauma layers. The very thing you wanted to protect gets sacrificed to the metric that was supposed to represent it. I have watched a coalition of small nonprofits spend eighteen months chasing a 'housing stability index' only to realize that stability, by their definition, meant tenants never moved—including when they got better jobs in safer neighborhoods. The proxy didn't just measure off. It punished the right outcome.

That order fails fast.

The donor who funded a proxy that backfired

A philanthropic foundation wanted to fund 'economic mobility' in a midsize city. Simple enough.

Pause here primary.

It adds up fast.

They chose a proxy: number of microloans disbursed per quarter.

That order fails fast.

Easy to count, easy to report, easy to celebrate. Two years and $4.7 million later, the microloan default rate hit 41%.

This bit matters.

Why? Because loan officers were incentivized to push money out the door—any door. Borrowers received capital with zero financial literacy support, zero follow-up, zero accountability. The proxy (loan count) had no guardrails. The donor pulled funding, but the damage was done: dozens of families saddled with debt they couldn't service, local trust in 'aid organizations' shattered for a generation.

'We measured speed because speed was easy. We forgot that the people we were measuring were not spreadsheets.'

— former program officer, after a post-mortem that nobody wanted to attend

The donor's intent was honorable. The mechanism was not. That is the cruel symmetry of bad proxy layout: good intentions make you blind to the seam where the proxy detaches from reality. You start asking 'how many?' instead of 'how well?' and the whole machine tilts.

The unintended consequences of easy metrics

Most units skip the hardest question: What does this proxy ignore? They grab what's available—attendance rates, survey completion numbers, social media shares—because the data is clean, auditable, and cheap. But cheap data has a hidden cost. A youth mentorship organization once celebrated '90% weekly check-in completion' as evidence of engagement.

It adds up fast.

Turns out the teenagers were tapping the 'done' button during breakfast to stop the reminder texts. The metric was pristine; the relationship was hollow.

Skip that move once.

The real cost surfaced six months later when attrition spiked—kids who never engaged quietly walked away.

Pause here primary.

The proxy gave false confidence, then delayed the signal of failure by half a year. That delay costs lives when the program serves homeless families or addiction recovery patients.

Easy metrics also distort internal behavior. Staff learn what gets counted. If you track 'calls made' instead of 'quality conversations held,' you'll get a call center that hits numbers and misses human beings. If you track 'food parcels distributed' instead of 'nutritional adequacy and cultural fit,' you'll move boxes while hunger persists. The proxy becomes the goal—and the original goal disappears into a spreadsheet. What usually breaks initial is trust: trust from beneficiaries who sense they're being processed, trust from donors who eventually smell the gap, and trust from your own team who know they're gaming the system but can't stop. That is the mess you inherit when you choose a proxy without asking whose story it will tell—and whose story it will erase.

According to field notes from working teams, the long-form version of this chapter needs concrete scenarios: who owns the handoff, what fails first under pressure, and which trade-off you accept when budget or time tightens — that depth is what separates a checklist from a usable playbook.

Prerequisites: What You Must Settle Before Choosing a Proxy

Clarify your theory of revision before you touch a metric

Most groups skip this. They grab the easiest observable proxy—website visits, survey completion rates, training certificates issued—and call it impact. That hurts. Without a clear causal story, your proxy becomes a vanity number. Draw a map: If we do X, then Y shifts, which eventually leads to Z. Write it down as a sentence, not a logic-model diagram. I have seen social enterprises track 'meals served' for years, only to discover their real adjustment was nutritional diversity—meals served hid the fact that people were eating the same starch every day. The proxy felt safe. It betrayed the long game.

The tricky bit is that most theories of adjustment are too vague. 'We empower women.' Great—but through what mechanism? Increased income? Decision-making power inside the household? Access to credit? Each mechanism demands a different proxy. Income proxies fail if the real revision is agency; agency proxies fail if the program's actual lever is cash flow. You must decide which link in the chain you are willing to bet on. That decision is ethical—it determines whose story gets counted and whose gets left out.

'A proxy that measures activity instead of outcome is not a shortcut. It is a blindfold.'

— bench notes from a monitoring & evaluation lead, after a three-year food-security program found no nutritional gain despite peak distribution numbers

Define the counterfactual—what would have happened anyway

Here is where ethics get sharp. If you pick a proxy like 'children enrolled in after-school tutoring,' you must ask: enrolled compared to what? The baseline dropout rate? The school's historical attendance? A matched peer group that received no tutoring? Without a clear counterfactual, your proxy cannot distinguish between your program's effect and a rising tide. I watched a literacy nonprofit celebrate 'books distributed per child' for two years. Then a local government started a universal reading campaign—distribution numbers soared, but the proxy could not tell whether the nonprofit or the policy drove the adjustment. They kept funding the flawed thing.

Most groups hate this stage because it feels like speculation. 'We cannot run a randomized trial—we are a small org.' Fair. But you can still name your counterfactual explicitly. We assume that without us, 60% of participants would not have accessed this service within 12 months.

Pause here initial.

State it. Share it.

That order fails fast.

Then concept your proxy to test that assumption, not just to confirm the program exists.

Skip that move once.

The catch is that ethical proxies often carry higher measurement costs—longer timelines, comparison groups, qualitative checks. You have to decide: is it ethical to use a cheap proxy that could be off?

Agree on what 'adjustment' looks like to different stakeholders

Your funder sees change as a percentage-point increase in a standardized test. Your floor staff sees change as a mother saying 'my child asks to go to school now.' Your participants may see change as dignity—being asked their opinion before a program is redesigned. If you pick a proxy that only serves one view, you are imposing a definition of success. That is an ethical choice, whether you admit it or not. I have seen programs collapse because the board's proxy (loan repayment rate) clashed with the community's real priority (financial resilience during a crisis). Repayment rates looked great; families were skipping meals to pay the loan. The proxy lied.

Run a quick alignment exercise before you finalize anything. List three stakeholder groups. For each, write one sentence answering: What would convince you this program worked? Compare the answers.

Fix this part opening.

If they conflict—and they will—that friction is your concept space. Not a problem to solve, but a tension to hold.

Pause here primary.

The proxy you choose should reflect that tension, not erase it. Otherwise, you build a measurement system that tells a comfortable story while the actual change slips away.

off order kills impact. Settle the story, the counterfactual, and whose definition counts—then you can pick a number.

Core Workflow: How to Evaluate a Proxy Step by Step

An experienced operator says the trade-off is speed now versus rework later — most shops lose on rework.

Step 1: Map proxy to outcome pathway

Pull out a whiteboard or a napkin — something you can shred later. Draw the line from what you can count today to the actual result you care about in three years. Most teams skip this: they grab a metric that is easy to track and call it good. Wrong order. You need a causal chain, not a correlation. If your proxy is 'number of training sessions completed' and your outcome is 'safer workplace behavior', trace every link. Does completion mean comprehension? Does comprehension survive a stressful shift? I have seen organizations celebrate 1,000 logged sessions only to discover injury rates climbed. The seam blew out because the proxy measured attendance, not absorption. That hurts. So force yourself to write out: if this proxy moves, then that intermediate state changes, which eventually shifts the long-term goal. If any link feels like a prayer, you have not found a proxy yet.

Step 2: Check for incentive distortions

A proxy is a contract written in numbers. People will read it and ask: what does this reward? If your metric is 'calls handled per hour', the logical move is to shorten each call, transfer aggressively, and hang up faster. That might boost the proxy. It also burns customer loyalty — the thing you actually wanted. The catch is that ethical proxies must pass a simple test: would a rational actor, optimizing only this number, eventually betray the mission? Run that thought experiment. If the answer is yes, you need to either add a counterbalancing metric or scrap the proxy entirely. One fix I have used: pair a speed metric with a quality audit (random sample, scored blind). But even that can drift — auditors get tired, thresholds soften. That said, a paired proxy beats a solo one every time. The hard part is accepting that no proxy is cheat-proof; you are just making cheating expensive enough to feel not worth it.

Step 3: Test sensitivity to context

What works in a stable funding quarter can poison your data when conditions shift. Imagine your proxy is 'volunteer hours logged per month'. Fine in calm weather. Then a crisis hits — say, a local disaster — and suddenly volunteer hours spike. Your proxy screams success. But the hours are reactive triage, not strategic capacity-building. The long-game outcome (deep community resilience) actually suffered because the organization pulled people from planned projects. The proxy was too blunt to feel the difference. So ask: if the environment changes — budget cut, staff turnover, election year — does this proxy still point in the right direction? If it only works under ideal conditions, it has a shelf life. Document that expiry. Most teams do not; they ride the proxy until it misleads them, then blame the tool instead of the design.

Step 4: Validate with mixed methods

Numbers lie less when you cross-check them with stories. Not as decoration — as a detection mechanism. Pick ten cases where the proxy moved positively. Interview the people involved. Does the story match the score? If you see a 30% jump in 'referrals to mental health support' but every interview mentions that staff now fear the referral will hurt their performance review, your proxy is capturing compliance, not care. That is a tell. The quantitative line looks clean; the qualitative seam shows rot. You do not need a full mixed-methods lab — a structured debrief with three frontline workers can surface a distortion that a dashboard never will. One rule I follow: before you lock a proxy into quarterly reports, run one cycle of parallel validation. Track the proxy and run a lightweight qualitative check. If they agree, you have confidence. If they diverge, you have a fix to make before the data gets weaponized.

'A proxy that passes all four steps still needs annual re-evaluation. Context decays. So should your certainty.'

— practice note, impact design workshop, 2024

Tools and Setup: Practical Aids for Proxy Selection

Rubrics and scoring sheets

Most proxy choices fail not because the logic was wrong but because the team had no shared language to argue about trade-offs. A scoring sheet surfaces those arguments early. Build a simple matrix: list candidate proxies down the left, then columns for data availability, causal proximity, measurement cost, and vulnerability to gaming. Score each cell 1–5, but force a written justification for every 4 or 5 — no empty high scores. I once watched a nonprofit pick "number of training sessions attended" over "skill test pass rate" because attendance data was already in their CRM. The rubric caught the real cost: zero correlation with actual behavior change. That single row killed the choice before it wasted a year of tracking.

Don't treat the rubric as a once-and-done artifact. Revisit it after your initial pilot. Columns that seemed obvious — like "ease of collection" — can shift when you discover that floor staff hate the new form. A living rubric is a negotiation tool, not a report card. The catch: rubrics tempt people to average scores and pick the highest total. That's lazy. Weighted columns or a simple veto rule (any proxy scoring 1 on causal proximity is dead) prevent math from overriding judgment.

Stakeholder feedback loops

The people who live inside your program every day will spot a bad proxy months before your quarterly review does. Build a loop that pulls their observations in, not one that waits for complaints. A short Slack thread each Friday — "What felt off about the data this week?" — catches the field worker who noticed clinic staff coaching patients on how to answer the satisfaction survey.

Most teams miss this.

That's not a data quality problem; that's a proxy-integrity problem.

That order fails fast.

The survey stopped measuring satisfaction the moment staff learned the scoring threshold. A feedback loop catches that seam before it blows out.

The tricky bit is trusting the loop. Managers often dismiss field reports as anecdotal noise. They aren't — they're leading indicators. Set a rule: any three independent reports about the same proxy distortion triggers a review.

Not always true here.

No exceptions. That hurts when the proxy is politically protected, but silence is worse.

This bit matters.

Pair the loop with a simple log: proxy name, reported issue, date, action taken. After six months, that log becomes your best evidence for which proxies need replacing.

"We ignored the field notes for two quarters. The proxy inflated our impact numbers by 40% — and we almost renewed the grant on that fiction."

— Monitoring manager, global health NGO, after a post-mortem

Pilot testing and data audits

A proxy that looks clean on paper often breaks in the real. Pilot testing is where you find the break. Run a small parallel test: collect both the proxy and a slower, more direct measure for a subset of cases — say, 50–100 observations. Compare the two. If the proxy drifts more than 15% from the direct measure, you have a signal that the proxy is measuring something adjacent, not the thing itself. Not yet a dealbreaker, but a warning that the proxy needs recalibration or replacement.

Data audits are the safety net. Schedule them every three months, not annually — annual audits only tell you what already went wrong. An audit checks three things: completeness (are we missing entire months?), consistency (did the definition of "active user" change between waves?), and outlier distribution (why did one site report zero dropouts?). Most teams skip this step because it feels like overhead. That's a mistake. The audit that found a coding error in our enrollment tracker saved us from reporting a 20% phantom improvement. Painful to run. Cheaper than the alternative.

End the pilot with a decision threshold. If the proxy passes, scale it with a six-month review date. If it fails, you don't abandon measurement — you rotate to the next candidate on your rubric list. Concrete next action: block two hours next week to build your initial rubric draft. Pull in one field worker and one data analyst. Disagree openly. That friction is where ethical proxy design lives.

Variations for Different Constraints

A field lead says teams that document the failure mode before retesting cut repeat errors roughly in half.

Low-budget settings: single proxy vs. composite

Your monitoring budget is a shoestring, and you can track exactly one thing. The instinct is to pick the single easiest number — clicks, form fills, app opens — and call it done. That hurts. I once watched a nonprofit run a literacy program use "books distributed" as their lone proxy. Distribution hit targets, reading scores flatlined. They celebrated the wrong signal for eighteen months.

High-stakes accountability: third-party verification

— A sterile processing lead, surgical services

Rapid iteration: proxy for learning, not judgment

The pitfall here is subtle: teams start treating these learning proxies as if they were judgment proxies. "Our pilot showed a 12% lift in engagement — let's scale it." No. Learning proxies are lenses, not verdicts. Use them to spot a direction, then design a proper proxy before committing resources. The fastest path to ruin is conflating "looks promising" with "proven effective." Save the rigor for the long game. Until then, practice the humility of a short attention span. It will protect you from falling in love with your own early numbers.

Pitfalls and What to Check When It Fails

Goodhart's Law in action

You pick a clean metric—say, 'hours spent in community engagement.' Six months later, staff pad their logs. Meetings that used to be forty minutes stretch to sixty. The number looks great. The trust evaporates. That is Goodhart's Law: when a measure becomes a target, it ceases to be a good measure. I have watched nonprofits triple their 'hours' while the actual relationship depth flatlined. The fix is not to abandon proxies—it is to audit what the proxy incentivizes. Check for behavioral spillover. If people start optimizing the number instead of the outcome, your proxy is already poisoned. Swap it. Or, better, pair it with a qualitative check that catches the gaming before the quarterly review.

According to practitioners we interviewed, the trade-off is rarely about talent — it is about handoffs, and however confident you feel after the first pass, the pitfall shows up when someone else repeats your shortcut without the same context. That one choice reshapes the rest of the workflow quickly.

Proxy decay over time

The context shifts. That brilliant proxy you chose in January—'number of repeat donations'—assumed a stable donor base. Then a crisis hits. New one-time givers flood in, repeat rate drops, and your proxy screams failure. But the program actually expanded reach. The seam blows out because the proxy's assumptions decay faster than the outcome changes.

Fix this part first.

Most teams skip this: they treat proxy selection as a one-and-done decision. Wrong order. You need a decay calendar—a simple note on when to re-evaluate the proxy's relevance.

Skip that step once.

Every six months, ask: 'Is this still measuring what we thought?' If the answer wobbles, retire it. Not yet? That hurts, but holding a stale proxy long enough lets you mistake drift for progress.

Unmeasured side effects

A youth program uses 'grade improvement' as its change proxy. Scores rise. Celebration. Meanwhile, teachers report that kids are skipping lunch to attend tutoring—hungry, stressed, burning out. The proxy caught one signal and blinded you to the damage. Unmeasured side effects are the quiet betrayers. What usually breaks first is the thing you decided was 'too hard' to track. Fixing this means adding a low-cost shadow metric: a single open-ended question in your weekly check-in. 'What else changed?' No dashboard. Just a Slack thread or a sticky note. I have seen one question rescue a whole evaluation design because someone finally said, 'The proxy works, but the kids hate it now.' Listen to that.

'A proxy that survives a year without adjustment probably isn't measuring change—it's measuring your comfort with the status quo.'

— field note from a program director who burned three proxies before admitting the fourth was also wrong

Stakeholder gaming

The clever ones figure out your proxy before you do. A microfinance organization tracked 'number of loan applications approved' as a proxy for financial inclusion. Loan officers started approving borderline applicants to hit targets—defaults spiked, families got trapped in debt. The proxy betrayed the long game because the stakeholders (loan officers, branches) had incentives to pump the number. Corrective action: build in a friction check. Add a second proxy that the first proxy's gamers cannot easily manipulate—like repayment rate at twelve months.

That is the catch.

If the two diverge, you have a gaming problem, not a measurement problem. That divergence is your early warning. Don't ignore it. Drop the first proxy. Or make it a trailing indicator only, never a bonus target. The incentive design is the ethics of the proxy.

The final check: ask yourself who benefits when the proxy looks good but the outcome does not move. If the answer is 'the person reporting it,' you already know what to fix. Replace it. Not next quarter—now.

FAQ or Checklist: Quick Reference for Ethical Proxy Design

What proxy should I avoid at all costs?

Vanity metrics. The ones that make your board nod but tell you nothing about whether anyone's life actually shifted. I once watched a team track 'program reach' — number of people who opened their email — and declare victory. Meanwhile, the community they claimed to serve had stopped reading after subject line two. That's not a proxy; that's a narcotic. Avoid any metric that only moves up. A healthy proxy should sometimes disappoint you — that's how you know it's measuring real friction, not your own echo.

How often should I reassess my proxy?

Every six months. But also: every time your context fractures — new funding source, staff turnover, a policy change in the region you work in. The trap is treating proxy selection as a one-and-done. It is not. We fixed this in my own work by scheduling a thirty-minute 'proxy autopsy' every quarter. We ask: Is the signal still truthful? Who is gaming this? What did we miss? Usually the answer is: 'The proxy is still okay, but the weights shifted.' That matters. If your proxy rewards speed but you suddenly need depth, the long game evaporates.

"A proxy that cannot be wrong about anything useful is a proxy you should fire before lunch."

— overheard at a nonprofit strategy retreat, after three years of misallocated budget

What if the only feasible proxy is flawed?

Use it, but build a sentinel. Do not let a weak proxy run unattended. Pair it with a qualitative check — four phone interviews a month, a public comment box, a random audit of five cases. The flaw is not the problem; the silence around the flaw is. I have seen teams run a deeply imperfect proxy (self-reported satisfaction) for two years because it was cheap. They caught nothing until the drop-off became a crater. The fix: treat the flawed proxy like a leaky thermometer — you still read it, but you also touch the patient. Set a rule: If the proxy improves but your qualitative story worsens, stop and dig. That tension is the whole point of ethical design.

One last check before you ship: map your proxy against your mission statement. If your proxy could stay green while your mission drifted into irrelevance, scrap it. Honest proxies hurt sometimes. That pain is the cost of knowing whether you are actually winning the long game — or just dressing up the short one.