Signs a programme's impact numbers don't add up

When a youth programme claims it's changed 500 lives this year but can barely name three participants, or when impact numbers climb sharply while funding stays flat, something's broken. Funders and community leaders spot red flags too late—after months of partnership or halfway through budget allocation—because they didn't press on the numbers when they mattered.

The commonest failure is counting the same person twice. A young person might attend a workshop, a follow-up session, and a mentoring call; three "participants reached" becomes nine on an annual report. Other times, an organisation counts everyone who walked through the door, regardless of whether they stayed five minutes or five weeks. The gap between headcount and actual engagement is where credibility dies.

When activity becomes theatre instead of change

A programme says it ran 20 workshops for 200 youth. But when you ask for attendance registers, the numbers are vague: "roughly 10 per session" or "about this many." Ask for the actual registers—the signed sheets or a digital roll call. If they don't exist or are scrambled, assume the figure is soft. Soft numbers are not lies exactly, but they're not truth either.

Watch for outcomes that are impossible to verify. "80% gained confidence" or "improved self-esteem in 95% of participants" rely on how you'd measure confidence. Unless the organisation conducted a baseline test at the start and a follow-up test after the programme, with a control or comparison group, these claims are hunches dressed as data. Ask: what tool did you use to measure this? How did you test it before and after? Who validates the result? If the answer is vague, the outcome claim is vague.

Another failure mode is scope creep in definitions. A programme designed to teach job skills might count someone "job-ready" if they attended one afternoon session. It might claim "employment outcomes" for people who applied for jobs but weren't hired. Read their definitions carefully. Are they what you'd reasonably expect, or are they stretched?

Costs per participant can reveal dysfunction too. If the organisation claims to serve 1,000 young people on a budget of R200,000, that's R200 per person—likely bare-bones. If the same organisation says it's now reaching 5,000 youth without a larger budget, the unit cost has halved. How? Unless they've genuinely scaled (new staff, partnerships, efficiency gains they can explain), the numbers probably inflated instead.

What to ask when numbers don't pass the sniff test

Request a participant breakdown by programme, site, and month. A credible organisation can show you not just totals but the shape of the data: how many people completed the full programme, how many dropped out, at which stage. Dropouts aren't shameful—they're honest. If an organisation claims zero dropouts or suspiciously low ones, they're either selecting only easy participants or not tracking properly.

Ask for a participant database or list (anonymised, of course) that proves the numbers. If they say they have "about 300 registered" but can't pull a list, they don't have 300 registered. Ask how long participants stay engaged and what happens after the programme ends. Do they follow up? Can they show contact details for a sample of past participants you could verify independently? This sounds intrusive, but it's what serious funders do.

Compare their latest figures to last year's. Did reach grow? By how much? Did quality metrics (completion rates, satisfaction, outcomes) stay stable or improve? If reach tripled but completion dropped by half, something cracked under the strain. If both grew, ask what actually changed—more staff, better marketing, new partnerships—to understand whether growth is sustainable.

The organisations worth backing know their numbers are scrutinised. They keep clean records, own their limitations, and don't massage definitions. When you're vetting potential partners on Strove or elsewhere, start by asking for their last year's numbers—not promises, actual data—and then ask the follow-up questions that separate real work from theatre.