<aside> 📌
Framework version of a standard I wrote and ran at Tandem Care. Client, employee, vendor, system and territory identifiers removed.
</aside>
I built an AI-assisted oversight layer over a fully remote home care operation: 114 active clients at peak, 200+ caregivers and 7 territories. It watched for missed follow-ups, money variances and coverage gaps. It worked, and then it began confidently telling me things that were not true.
This is the standard I wrote in response.
<aside> 💡
The design constraint most people miss: the check has to work by format, not by knowledge. I couldn't verify the content of a list about forty caregivers, because checking content requires knowing every case, which is the entire reason I asked for the list.
</aside>
Four findings were wrong in a single weekend review, all the same way: the system read a chat message, a shift summary or its own earlier output and stated it as if it were the record. Two later errors nearly became accusations against people who had done their jobs correctly.
| What the AI said | What was actually true |
|---|---|
| "This coordinator never replied to the client." | She replied in 22 minutes. The client wrote from one phone number and she answered on their second. The query filtered by number, so it saw the question and not the answer. |
| "A large batch of outbound messages have no owner." | They all had owners. A blank agent field is the signature of a reply sent from the CRM, not the absence of one. |
| Failure mode | What it looks like | The fix |
|---|---|---|
| Absence of evidence reported as evidence of absence | "Nothing there" from one empty check | Every absence has a second place that must also be empty. One empty check is not a finding. |
| Querying the wrong key | Assuming a field means what its name suggests | Probe the shape of the data before reasoning about it. |
| Fluency outrunning verification | A plausible number or time filled in when the real one wasn't available | Every number and time carries its source inline. |
| Trusting its own earlier output | A count quoted from a previous message | Re-derive it from the source, every time. |