Every floor that turns on automated call review has the same first week. The queue fills up, the supervisor works through it in a panic, and somewhere around day four somebody asks the question the whole programme actually turns on: what am I supposed to do with these?
It is a fair question and it is usually unanswered, because the vendor sold detection and the buyer assumed the rest was obvious. It is not obvious. A flag is a machine saying a human should listen to this moment — it is a pointer, never a verdict, and the distance between those two things is where a QA programme either becomes useful or becomes a liability.
This is the loop we would run. Steal it, argue with it, but write down whatever you settle on, because an undefined loop reliably degrades into no loop.
The four outcomes, and nothing else
Every flag should end in exactly one of four states. Keeping the list this short is the point; the moment a fifth informal state appears — "watching it", "mentioned it" — the queue starts to rot.
Not an issue. You listened, the flag was wrong or the context explains it. Close it with one line saying why. These are not failures of the system; a rubric that never over-flags is one that is missing things.
Coaching. Real, but a skills or habit problem rather than a breach: a disclosure delivered so fast nobody could parse it, a tone that escalated, a promise phrased loosely. Outcome is a conversation with the collector and a note recording that it happened.
Documented incident. Something that touches the statutory obligations — third-party disclosure, contact after a cease request, a misrepresentation. Outcome is a written record, whatever remediation your policy specifies, and a decision about whether the consumer needs to be contacted. This category has a different retention life from the other three and should be searchable on its own.
Escalation. Beyond a supervisor's authority — a pattern rather than an instance, a client or creditor implication, anything a compliance officer or counsel needs to see. Outcome is a handover with the audio, the transcript segment and your reading of it.
The discipline is that every flag gets one of the four, and the closing note says which and why. A supervisor who closes twenty flags with "reviewed" has produced a record proving they opened the queue and nothing else.
How long you have
Set a clock and publish it. Ours would be:
| Flag category | Triaged within | Closed within | Who decides |
|---|---|---|---|
| Statutory: cease, third-party disclosure, threat, misrepresentation | same business day | 3 business days | Supervisor, escalating on pattern |
| Missing or incomplete disclosure | 1 business day | 5 business days | Supervisor |
| Tone, professionalism, script adherence | 3 business days | next coaching cycle | Supervisor |
| Repeat of a previously coached item | same business day | 3 business days | Supervisor plus compliance |
Two things make these numbers workable rather than aspirational. The first is that review is next-morning by design, so the clock starts on a fixed cadence rather than whenever somebody gets to it — the daily report lands before the shift and the day's triage is the first job, not the last. The second is that most flags are not statutory; if your queue is mostly top-row items, the timings are the least of your problems.
The note is the deliverable
The thing that survives is not the flag. It is what a supervisor wrote next to it, and most notes are written for the person writing them rather than for a stranger reading them in eighteen months.
A usable note has four parts:
- What was heard, quoted, with the timestamp. Not "collector was rude" — the actual words and the second they occur at.
- The reading. Why this is or is not what the flag suggested. This is the part only a human can supply and the part most often skipped.
- What was done. Coached on this date, incident logged under this reference, escalated to this person.
- What should happen next, if anything, and when it will be checked.
Four sentences is usually enough. The scorecard piece makes the same argument about evidence attaching to findings, and for the same reason: a conclusion nobody can retrace is a conclusion nobody can defend.
Write in the past tense about facts and be explicit about uncertainty. "Unclear whether the consumer said stop calling or stop mailing; escalated" is a far better note than a confident guess, and it is the note that protects everybody if the answer turns out to matter.
The clean call that gets flagged, and the ugly call that does not
Two failure modes deserve naming because they behave differently.
The false positive. A phrase that reads as a threat in text and is plainly not in audio. Sarcasm. A consumer quoting somebody else. These should be closed fast and cheaply, and the rate should be tracked: if a category is producing mostly nothing, tune it, but tune it deliberately rather than by switching it off in irritation on a busy Friday.
The false negative. Harder, and the reason the human loop exists. A call can be entirely clean on every rubric dimension and still be a problem — the collector was reading the wrong account, the promise-to-pay was recorded against the wrong balance, the consumer was confused and never said so. No transcript reveals that on its own, which is why the desktop side of supervision exists alongside the call side and why we treat them as one picture rather than two products — the argument made at length in supervising collectors who work from home.
So the loop needs a way in that does not start with a flag: a supervisor's own spot checks, a collector raising something, a complaint arriving from outside. If the only door into the QA record is the machine's, the record inherits the machine's blind spots.
The queue nobody works
This is the part vendors do not say out loud, so we will.
A flag queue that sits unworked is not neutral. Before automated review, a floor could truthfully say it did not know about a problem call. Afterwards, the record shows the moment was surfaced and shows what happened next — and "nothing" is a legible answer. The Consumer Financial Protection Bureau publishes supervisory materials describing expectations around monitoring and corrective action, and corrective action is the half most easily left undone. The Federal Trade Commission has enforced in this area for decades and its case material is worth reading precisely because so much of it concerns conduct that somebody internally could have seen.
We are not saying detection creates liability — the underlying conduct is the liability, and it was there before anyone had a transcript of it. We are saying that buying visibility commits you to acting on it, and a floor that cannot staff the triage loop should size its programme to what it can actually work rather than switching on everything and hoping.
Practically, that means: define who owns the queue by name, cap what one person is expected to close in a day, and review the age of the oldest open flag at the same meeting where you review everything else. An oldest-open-flag number that keeps climbing is the single best early warning that the programme is decorative.
What automation should never decide
Three decisions stay with a person, permanently, and we build to that:
Whether a statutory line was crossed. The rubric reports per dimension — was the debt-collection disclosure given, was a cease request or dispute raised, what was the tone on each side — with the quote and the timestamp that triggered each. It does not average those into a single grade, because a warm, well-handled call with a violation inside it passes on a blended score. It also does not render the verdict.
Whether a person is disciplined. A pattern in the data is an input to that conversation and never the conversation itself.
Whether a consumer gets contacted about what happened. That is a policy and often a legal question, and it belongs with the people who can be held responsible for the answer.
As of September 2026
Category definitions and the timings above are our recommended starting point as of September 2026, not a standard published by anyone. The underlying rules — the Fair Debt Collection Practices Act and Regulation F — are public at the Electronic Code of Federal Regulations, with rulemaking history at the Federal Register. Where a genuine legal conclusion is needed, this stops and your counsel starts.
If you want to see what the flags and the daily report look like against your own floor before committing to a loop, tell us how you dial and how many collectors you have — and see how it works for the mechanics of what gets captured and when.
Frequently asked questions
What does a supervisor actually do with a flagged call?
Listen to the flagged seconds, then close it into one of four states: not an issue, coaching, documented incident, or escalation — with a note saying which and why. The note should quote what was heard, give the reading, record what was done and say what happens next.
How fast should flags be worked?
Statutory categories — cease requests, third-party disclosure, threats, misrepresentation — same business day for triage and a few days to close. Tone and script items can wait for the next coaching cycle. Publish whatever you choose, because an unpublished target is not a target.
Are all the flags real problems?
No, and a system that produced only true positives would be one tuned so tightly it was missing things. Expect a proportion to close as nothing on inspection, track that rate per category, and tune deliberately rather than switching categories off when the queue looks heavy.
Does the software decide whether a rule was broken?
No. It reports each dimension separately with the quote and timestamp that triggered it and points a human at the moment. The judgement, the discipline decision and any consumer communication stay with people who can be held responsible for them.
What happens if nobody works the queue?
The record then shows a moment was surfaced and that nothing followed, which is a worse position than a smaller programme worked properly. Assign the queue to a named person, cap the daily load, and track the age of the oldest open flag.
Can a clean-scoring call still be a problem?
Yes, and it is the case people underrate. A collector reading the wrong account while agreeing a payment scores clean on every conversational dimension. That is why desktop activity sits beside the call record instead of being sold separately, and why the loop needs doors in that do not start with a flag.
Reviewing two percent of your calls?
CollectionsQA records, transcribes and scores every call your collectors take, and sends one supervisor report each morning naming the calls that need a human.