Seven Numbers a Collections QA Programme Should Report, and Three It Should Stop Reporting

Most collections QA reports open with the same line: an average quality score, somewhere in the high eighties or low nineties, up or down a point on last month. Everybody in the meeting nods at it. Nobody in the meeting can say what they would do differently if it had come in two points lower.

That is the test for a QA metric. If the number moved, would somebody on the floor change what they do tomorrow? An average score fails it. So do several other numbers that fill QA decks because they are easy to produce.

This piece sets out seven numbers that pass the test, works them through on an example floor, and names three that should come out of the report. None of it needs statistics beyond division, with one exception near the end that is worth the five minutes.

The example floor

Twenty collectors, each handling about 60 connected calls a day, 21 working days in the month. That is 20 × 60 × 21 = 25,200 calls. Every recorded call is transcribed and screened against a collections rubric, and a supervisor works each flag to a decision the next morning: confirm it as a finding, or dismiss it with a note.

Here is the month, by category.

Category	Flags raised	Confirmed findings	Confirm rate	Findings per 1,000 calls
Disclosure missing or late	310	96	31%	3.81
Third-party disclosure	88	9	10%	0.36
Cease or dispute raised, not handled on the call	204	28	14%	1.11
Threat or misrepresentation language	96	11	11%	0.44
Abusive language by the collector	58	7	12%	0.28
All categories	756	151	20%	5.99

Everything below is read off this table or off the review log behind it.

1. Coverage: what share of calls did the programme see?

Coverage is calls screened divided by calls made. It is the first number because every other number is conditional on it: a findings rate computed on a slice of the floor describes the slice.

The trap is the denominator. The QA system knows how many calls it processed; it does not know how many calls happened. That figure has to come from the dialler or the phone system, and the two have to be reconciled each month. On our example floor the dialler logged 25,200 connected calls and the QA record holds 25,200, so coverage is 100 percent. If the QA record held 21,400, the report should say 85 percent and then say which 15 percent is missing — one collector's machine, one campaign, every call made from a desk handset.

A sampled programme reports this number too, and honestly stated it is usually 1 to 3 percent. Why two percent sampling fails covers what that does to everything downstream.

2. Flag rate, by category

Flags raised per 100 calls: 756 ÷ 25,200 = 3.0 percent. On its own this says little about the collectors. It is mostly a property of the screen — how sensitive the rubric is — and its main use is workload planning. At 3 percent, a supervisor on this floor has about 36 flags to work each morning.

Watch it by category and over time. A flag rate that doubles in one category in one week, with no change in confirm rate, usually means something changed in the environment: a new script, a new campaign with a different consumer population, a transcription problem on one team's headsets.

3. Confirm rate, by category

Confirmed findings divided by flags worked: 151 ÷ 756 = 20 percent overall, but look at the spread. Disclosure flags are confirmed 31 percent of the time; third-party flags 10 percent.

Confirm rate measures the screen and the reviewers together, and it is the number that keeps both honest:

• Very low in a category (say under 5 percent, sustained) means supervisors are spending most of their time dismissing noise. The rubric wording for that category needs attention.
• Very high (say above 70 percent) is not automatically good news. It can mean the screen only catches flagrant cases and misses the marginal ones, or that reviewers have started confirming whatever arrives. Blind re-scoring is the check, and calibrating call reviewers describes how to run it.
• Drifting with no change on the floor points at the reviewers before it points at the collectors.

4. Confirmed findings per 1,000 calls, by category

This is the outcome number: 5.99 per 1,000 overall, 3.81 of which is the disclosure category.

Express it per 1,000 calls rather than as a percentage, for two reasons. The percentages are small enough to be unreadable (0.036 percent for third-party disclosure), and a per-call rate stays comparable when call volume changes. A floor that reports "151 findings this month, 139 last month" has told you nothing if volume rose 15 percent in between.

Keep the categories separate. A floor whose total fell from 6.4 to 5.99 per 1,000 looks improved; if the fall came entirely from disclosure while third-party disclosure rose from 0.2 to 0.36, the total hid the thing most worth a conversation. The same logic is why our own rubric reports each dimension on its own and never as a blended score — the argument is laid out in building a scorecard that survives a dispute.

5. Time to disposition

Two figures: the median age of a flag when a supervisor decided it, and the age of the oldest flag still open.

The median tells you whether the review loop is running. Next-morning review should produce a median under two working days. The oldest-open figure tells you whether anything has been abandoned, and it matters more than it looks. A flag that pointed at a possible problem and then sat untouched for seven weeks is a worse fact to have in your records than no flag at all. What a supervisor should do with a flagged call goes through the queue discipline; the metric is how you find out whether it is being followed.

6. Repeat rate after coaching

Of the collectors who were coached on a confirmed finding, what share had another confirmed finding in the same category within the following 60 days?

This is the only number on the list that measures whether QA changes anything. Findings that are counted, logged and filed without altering what happens on the phone are a cost with no return. If 14 collectors were coached on late disclosure in July and 9 of them had a late-disclosure finding again by the end of September, the repeat rate is 64 percent, and the conclusion is about the coaching, the script or the workflow — not about nine individuals.

It needs one thing most QA records lack: a dated note of the corrective step attached to the finding. Supervisory guidance from the Consumer Financial Protection Bureau describes a compliance management system in terms of monitoring and corrective action, and its supervision and examination materials are a useful read for how an outside examiner frames the question. A creditor client's audit asks the same thing less politely.

7. Reviewer agreement

When two reviewers decide the same flags independently, how often do they match on the ones at least one of them thought were real? This is a quarterly number rather than a monthly one, and the calibration guide linked above works through the arithmetic. It belongs on the report because every number from 3 to 6 passes through a reviewer's judgement, and the NIST/SEMATECH e-Handbook of Statistical Methods is blunt that data from an unchecked measurement system should not be trusted for decisions.

The five minutes of statistics: small numbers per collector

Floor-level rates on 25,200 calls are stable. Collector-level counts in rare categories are not, and this is where QA reports do the most damage.

Take third-party disclosure: 9 confirmed findings across 20 collectors in the month. Each collector made about 1,260 calls, so at the floor rate the expected count per collector is 1,260 × 0.36 ÷ 1,000 = 0.45 findings a month.

Now suppose all twenty collectors are exactly equally careful. Rare, independent events like this follow a Poisson distribution, and with an expected count of 0.45 the chance that a given collector shows two or more findings in a month is about 7.5 percent. Across twenty collectors, that is one or two people a month who will appear at the top of a "most third-party findings" league table through chance alone. Next month it will be a different one or two.

The practical rules that follow:

• Rank collectors only on categories with enough events. Disclosure, at nearly 5 expected findings per collector per month, supports comparison. Third-party disclosure does not.
• For rare categories, review every finding as an incident — what happened on that call, what the collector does next — and look for a pattern over a quarter, not a month.
• Never rank on a zero. A collector with no findings in a rare category has told you almost nothing; 64 percent of identical collectors would also show zero.

This is the mirror image of the sampling problem. A small sample cannot see rare events; a full record sees them all, and then tempts you to over-read each one.

Three numbers to take out of the report

The average QA score. It blends graded rows with compliance rows, so a warm, well-controlled call with a missing disclosure can score 91. It moves when the reviewer rota changes. And it compresses: on most floors nearly every collector lands within a few points of every other, which is why nobody acts on it.

Calls reviewed. As a count it measures supervisor effort, and it rewards reviewing short, easy calls. Replace it with coverage (number 1), which measures what the floor needed rather than what the reviewers did.

Percent of collectors "passing". A pass mark applied to a blended score inherits every problem of the score and adds a cliff edge. Two collectors a point apart on either side of the line are not meaningfully different, and one of them now has a documented failure.

We reported an average score to the owners every month for six years. The first month we replaced it with findings per thousand calls by category, the question from the top of the table changed from "is ninety-one good?" to "why is one team's disclosure rate triple the other's?" That was the first useful question QA had been asked.

— Director of compliance, third-party agency, 14 years, name withheld by request

What the rules have to do with it

Nothing in the Fair Debt Collection Practices Act or Regulation F prescribes a QA metric. The categories in the table are operational groupings a floor might build around the conduct the statute and regulation address — the FDCPA text and 12 CFR Part 1006 are the primary sources — and which categories your floor tracks, and how each is defined, is a decision for your compliance lead and counsel. A findings rate is a count of what reviewers confirmed against your own scorecard. It is not a count of violations, and nothing here is legal advice. For context on the size of the occupation these numbers describe, the Bureau of Labor Statistics publishes data on bill and account collectors.

Where our product fits, and where it does not

As of October 2026, our Voice tier — $99 per monitored collector per month, with supervisor and admin logins free — records, transcribes and screens every call that runs through the collector's computer, and logs each confirm or dismiss against the reviewer who made it. Flags arrive with the category, the timestamp and the quoted line. That log is the raw material for numbers 2 to 5 and number 7. Features describes each piece and pricing lists the tiers.

Be clear about what it does not hand you:

• The coverage denominator. We know the calls we captured. The count of calls made has to come from your dialler, and a call placed on a desk phone that does not route through the PC never reaches us at all.
• Account-level views. Calls tie to collectors and times, not to accounts in your platform, so "findings per placement" or "findings per client portfolio" needs your system of record alongside.
• Repeat rate. It depends on your own dated coaching notes. We hold the findings; you hold what was done about them.
• Anything same-day. Review is next-morning by design, so every number here describes yesterday and earlier.

If a monthly report built on these seven numbers is what you are trying to get to, tell us what you dial from and we will say plainly which of them our record supports on your floor.

Frequently asked questions

What is the single most useful collections QA metric?

Confirmed findings per 1,000 calls, reported separately for each category. It is an outcome rather than an effort measure, it stays comparable when call volume changes, and keeping categories apart stops an improvement in a common category from hiding a deterioration in a rare one.

Why is an average QA score a poor metric?

It blends compliance rows with graded soft-skill rows, so a call with a serious problem can still score well. It shifts when different reviewers are on the rota, and it compresses nearly every collector into a narrow band, which leaves nothing to act on.

How do we calculate QA coverage?

Divide the number of calls your QA process actually examined by the number of calls the floor made in the same period. Take the second figure from the dialler or phone system, not from the QA tool, and when coverage is below 100 percent, state which calls are missing and why.

What is a good confirm rate for flagged calls?

There is no universal target, and we would be wary of anyone who offers one. Read it as a diagnostic: a very low rate means reviewers are wading through noise, a very high one may mean the screen misses marginal cases or reviewers are rubber-stamping. Track it by category and investigate drift.

Can we rank collectors by number of findings?

Only in categories where each collector has enough events for the comparison to mean something. In rare categories, a collector can show two findings in a month through chance alone, as the worked example shows. Treat rare findings as individual incidents and look for patterns across a quarter.

How often should QA metrics be reported?

Monthly for coverage, flag rate, confirm rate, findings rate and time to disposition. Repeat rate needs a 60-day window after coaching, so it lags. Reviewer agreement is a quarterly figure tied to calibration sessions.

Does a low findings rate mean the floor is compliant?

No. It means reviewers confirmed few findings against your scorecard on the calls your programme saw. Whether the scorecard covers what it should, and whether any given call amounts to a violation, are questions for your compliance lead and counsel.