What a satisfactory answer sounds like
A flag you cannot close is a flag you will keep re-raising, so each one needs a written pass condition — the specific sentence from the specific owner that ends it.
You raised it well. Right owner, honest severity, a small proposal, an explicit limit. The owner replied within the day: thanks for flagging, we are on it.
Three months later you notice the same thing on a different surface and cannot remember whether the first one was ever resolved. So you raise it again, to the same person, who remembers the first conversation differently than you do. That is the failure this lesson prevents, and it is entirely preventable, because it is caused by a missing sentence you were supposed to write before you ever opened your mouth.
Write the ending before the beginning
A pass condition is the specific sentence, from the specific owner, that lets you mark the row closed. You write it when you write the flag, not when the reply arrives — because a pass condition invented after the fact will be shaped to fit whatever you were told, which is how “we are on it” becomes a closure.
The test for a good one is symmetrical, and both halves matter. It has to be specific enough that you would recognise it if somebody said it, and specific enough that you would notice if they said something adjacent instead. Most weak pass conditions fail the second half. “They confirm the data is handled properly” is satisfied by almost any sentence a reasonable person could say.
The standard the sentence has to meet
Simon Willison, writing about injection-detection filters years before anyone was shipping agents:
“When you’re engineering for security, a solution that works 99% of the time is no good.” (Simon Willison, 17 September 2022)
He is making a point about controls. The same test applies to answers, which is the move this lesson is built on. His contrast is exact: “There is a known, guaranteed to work mitigation against SQL injection attacks: you correctly escape and quote any user-provided strings…Trying to prevent AI attacks with more AI doesn’t work like this.” A guarantee is checkable by inspection. A probability is not, and neither is a reassurance.
So a satisfactory answer names something a third person could go and verify: a mechanism, an enumeration, a number, a document, or an accepted risk with a name against it. Anything else is a mood.
Accepted is a closure, not a defeat
The response options a risk framework lists — “mitigating, transferring, avoiding, or accepting” (NIST AI RMF 1.0, MANAGE 1.3) — include one that many architects refuse to count. The same framework asks that “negative residual risks…to both downstream acquirers of AI systems and end users are documented” (MANAGE 1.4).
Put those together and you get the most useful pass condition available to you, because it is the one an owner can always produce:
The named owner states, in writing, that this is accepted, and points at where it is recorded.
That closes the row. It does not require anybody to fix anything, agree with your severity, or admit a problem. It requires them to make the decision on purpose, which is the only thing you were ever entitled to ask for.
Pass conditions, in the shapes they come in
| Flag | The sentence that closes it |
|---|---|
| A recorder is running on a surface showing personal data | A stated masking default tied to the installed SDK major version, read from the vendor’s current documentation rather than remembered — plus which of inputs and page text it covers. |
| The trace view shows everything | Redaction happens before storage, or access is described in terms of roles. “It is internal only” describes the current audience, not a boundary. |
| No retention period for transcripts | A number, the document it lives in, and the role that owns it. “Indefinitely, deliberately, recorded here” passes. |
| Personal data crossing a border nobody named | A named transfer basis for each hop, rather than a claim about where data sits. |
| A telemetry vendor on an audited product | Whether it is treated as a subservice organization, and whether it is carved out or included. |
| Model output reaches an HTML sink | A named sanitiser applied at the sink, its configuration written down, and an answer for the browsers where the native API is not available. |
Every one of those is a sentence somebody can actually say, and every one of them fails loudly if the reply is adjacent. That is the property to aim for.
Four answers that are not closures
- “We sanitise the prompt.” It describes a control class current OWASP guidance records as insufficient, and no reply is worse than a correction. Ask what happens when it misses one, and how anyone would know.
- “It is internal only.” A description of today’s audience. It is not an access boundary and it changes the week somebody shares a link.
- “We are SOC 2.” A report about controls, over a period, against categories somebody chose. It is not an answer to a specific question about a specific surface, and it may carve out the exact vendor you asked about.
- “We discussed it and we are comfortable.” The closest to a real closure, and the most dangerous for that reason. Comfort is not a record. The fix is a single question rather than a challenge: could you put that in the thread so I can close it?
Where people get burned
There is one pass condition you must never write: a demand that somebody prevent prompt injection. OWASP’s 2026 position is that no reliable prevention mechanism exists and that defence has to be architectural rather than interceptive. A flag whose closing sentence requires a fix nobody has will stay open forever, and it will teach the room that your flags are unclosable by construction. Ask for a constraint, an enumeration, a named owner or an accepted risk. Those are things an organisation can actually produce.
Check your recall
Answer from memory — no scrolling back.
Retrieval check
You wrote a pass condition and the owner answers something adjacent but genuinely better than what you asked for. Now what?
Check your answer
Close the row and rewrite the pass condition to what they said. The condition is a tool for recognising an ending, not a contract they agreed to. Insisting on your original wording when the substance is answered is exactly the behaviour that gets an architect labelled as the blocker.
The thing worth checking first is whether it is genuinely better or merely different. The test is the same one: could a third person verify it? If yes, take it, record what they actually said rather than your paraphrase, and move on.
Hands on
Close the log
Done when: Every row in FLAG-LOG.md has a written pass condition specific enough to recognise and specific enough that an adjacent answer would fail it. No row asks for prompt injection to be prevented. No owner column says unknown. At least one row is marked raised, with a date.
- Go row by row and write the closing sentence, in the owner’s voice, before you know what they will say. Present tense, first person, as if they said it.
- Run the adjacency test on each. Write down the nearest plausible answer that should not close it. If you cannot think of one, the condition is too loose and needs a mechanism, a number or a document in it.
- Check every condition for a demand nobody can meet. Anything requiring prevention, a guarantee, or the absence of risk gets rewritten as a constraint, an enumeration, or a documented acceptance.
- Look at the owner column across the whole file. Any row still saying unknown after this module is the signal the course did not land on that row, not that the row was hard. Go back and run the closing-sentence test on it again.
- Pick the single flag you would raise first in a real meeting, and mark it
raisedwith the date once you have raised it. One actually raised flag is the transfer this whole course is for. - Bring the log in. I will push on any pass condition that a merely polite answer would satisfy, and on any row where the owner and the closing sentence do not match.
What this does not cover
This is internal escalation: you, inside one organisation, raising something to a named colleague, with no publication step anywhere in it. Reporting a vulnerability you found in somebody else’s product is a different act with a different lifecycle — an external reporter, a vendor, and a decision about public disclosure — and it has its own standards and its own norms. This course does not teach it, and importing its vocabulary here would describe a mechanism that is not the one you are in.
The course ends here, and the ceiling holds to the last line: nothing in it qualifies you to own a control, write a policy, or judge whether a mitigation is sufficient. What you have is a set of things you can recognise, a role to send each one to, and a sentence that ends it. The flags reference at the end of this course is built to be reopened before a meeting rather than read once, and the log stays honest the same way it started — a blank cell is information, and an invented owner is not.
Read this next — primary source
You can’t solve AI security problems with more AISimon Willison, 17 September 2022 — free, about five minutes, live 2026-09-05. An independent engineer with no product in this category.
This lesson takes one standard from it and generalises: “When you’re engineering for security, a solution that works 99% of the time is no good.” That is a test for answers, not just for filters — an answer you cannot check is a probability wearing a suit. Read it in full; it is short, and the contrast it draws with SQL injection escaping is the clearest available illustration of the difference between a guarantee and a hope. Where it stops: it is about one control class, and it offers nothing about closing a governance flag. That part is this course’s own.
Stuck, curious, or think this lesson is wrong? Ask your teaching agent. The lessons are the scaffold; the conversation is where the learning gets unstuck.