PolicyForge — Real options, not one draft

The User Moment

I used to think the hard part of an AI writing tool was getting it to write. It’s not. The hard part is getting it to know when not to write with confidence.

Picture the moment a customer sits down with PolicyForge under deadline pressure: a renewal notice on the desk, a policy that needs to map cleanly onto a compliance framework by a fixed date. They ask the tool to draft a clause. What they actually need isn’t just words on a page; they need to know, at a glance, whether this particular answer is one they can trust as-is or one that needs their eyes first. A tool that answers every question with the same flat, confident tone is worse than useless in a domain like compliance, it’s actively dangerous. A wrong answer delivered with total confidence is exactly the kind of thing that gets rubber-stamped by someone in a hurry, and that’s precisely the failure mode PolicyForge exists to prevent.

So the promise made publicly, a compliant policy in minutes, turned out to hide a much harder design problem underneath it: minutes, yes, but also right, and also honest about when it isn’t sure it’s right. Those three requirements don’t naturally travel together. Speed tempts you toward a single confident pass. Correctness tempts you toward slowing down and checking everything by hand. Honesty about uncertainty tempts you toward hedging so much that nothing reads as usable. Getting all three into one experience meant none of them could be treated as the “default” and the others as afterthoughts bolted on later.

I spent longer than I expected just sitting with that tension before writing a single line of the actual generation logic. It felt important to name it clearly, because every subsequent decision, the prompt structure, the routing logic, the privacy rules, traces back to trying to resolve it.

The Design Problem

Compliance mapping isn’t one kind of task pretending to be three, it’s genuinely three different situations wearing the same interface. Sometimes an organization’s practices map cleanly onto exactly what a framework requires, and there’s nothing to hedge about. The facts fit, the requirement is well understood, and the honest answer is a clean, confident clause. Sometimes there’s a real gap: the organization doesn’t do the thing the framework wants, and the honest answer isn’t a polished clause at all, it’s a clear explanation of what’s missing and what to do about it. Handing someone a smooth-sounding clause when the underlying practice doesn’t actually exist yet isn’t help, it’s a liability waiting to surface during an audit. And sometimes the situation looks fine on the surface, but past experience says this exact type of clause tends to get kicked back for rework, which means the honest answer needs a caveat attached, not a clean, confident sentence.

Treating all three of those as the same output format was the mistake to catch early, and it very nearly wasn’t caught early. A single rigid draft has no way to distinguish “this is done” from “this needs your eyes.” Everything reads the same. Everything gets skimmed the same way. And a reviewer who’s been trained by twenty confident-looking outputs to stop reading closely is exactly the reviewer who misses the one wrong answer buried in output twenty-one. The irony is that the more polished every output looks, the less scrutiny each one receives, which means polish without honesty is actively self-defeating in this particular domain.

There was a second problem sitting right underneath the first, and it’s the one I stressed over more: none of this “smarter generation” mattered if the cost of it was exposing an organization’s private information along the way. Every prompt sent to generate a clause carries context: organizational facts, relevant framework language, details about how a team currently operates. What it must never carry is anything that identifies a specific person tied to that account. Not a name, not an account identifier, nothing that would let generated content, logs, or any downstream caching layer become an accidental record of who works at a given company or what their personal details are. A generation engine that’s clever about confidence but careless about privacy isn’t a product worth shipping, no matter how good the drafts read.

The Options

The obvious path was to build the simple thing: take the organization’s facts, take the framework requirement, ask a model to write the clause, done. One shot, one draft, ship it. That fails in a predictable way. It has no mechanism for distinguishing confidence levels, so every clause looks equally trustworthy regardless of whether it deserves to be. It’s also the fastest option to build, which makes it tempting precisely when I need to be most suspicious of it.

A second option was to let the model decide, case by case, whether to hedge its own answer, essentially asking it to self-assess its confidence in the moment and adjust its tone accordingly. I considered this seriously, because it’s the more flexible-sounding approach and it would have required less upfront design work. But a model deciding in the moment whether to caveat is a model you can’t fully audit after the fact. You can’t point to a rule and say “this is why it flagged a warning here.” That’s an unacceptable answer to give a compliance-conscious customer who wants to know why their review queue is organized the way it is.

A third option, one I didn’t take seriously for long, was to just always hedge. Attach a caveat to everything, flag every clause as needing review, and let the human sort out which flags actually matter. This technically solves the audit-trail problem, but it destroys the actual value of the product. If everything is flagged, nothing is, and the customer is back to reading twenty identical-looking warnings with no signal about which one actually deserves their time.

On the privacy side, the obvious-but-wrong option was to rely on careful prompt-writing discipline: trust that whoever builds a given generation call remembers to leave personal fields out of it. I rejected that almost immediately. A rule that depends on someone remembering something correctly, every single time, under deadline pressure, isn’t a rule. It’s a hope, and hope is not a security control.

The Build

What I landed on: the decision about which kind of answer to generate isn’t left to the model’s judgment in the moment. It’s driven by the same signals that already rank which gaps matter most: how well an organization’s stated facts match a given framework requirement, how critical that requirement is, and whether similar clauses have needed correction before. Those signals decide, ahead of time and deterministically, which of three behaviors the engine should produce for a given clause, before generation even begins.

When the facts line up cleanly with a well-matched, high-priority requirement, the engine writes a direct answer with minimal hedging. It earned that confidence through the ranking signals, not through the model’s own say-so. When there’s a real gap between what the organization does and what’s required, the engine doesn’t paper over it with a vague clause; it produces a structured explanation of exactly what’s missing, framed as something to act on rather than something to sign off on. And when the historical record shows that this type of clause has previously needed rework, the engine is required to attach an explicit caveat and a visible confidence flag. It isn’t allowed to quietly suppress that warning just because the draft otherwise reads well. That last rule is the one I’d fight hardest to defend if challenged on it, because it would be easy, and tempting, to let a good-looking draft speak for itself instead.

On the privacy side, the rule locked in is blunt on purpose: certain categories of information are simply not allowed into a generation prompt, full stop, enforced at the code level rather than left as a guideline someone might forget under deadline pressure. If a piece of data doesn’t belong on the approved list, the system errors out rather than silently passing it through. I didn’t want a rule that depended on someone remembering to redact something correctly every single time. I wanted a rule that made the wrong path structurally unavailable.

The same logic extends to a performance layer that reuses previous responses for speed. The moment any personal-data field shows up anywhere in a request, that reuse turns off automatically for that request. No exceptions, no judgment call, no “just this once because it’s a small field.” Speed loses to privacy every time that conflict comes up, by design, not by discipline someone has to remember to apply.

There’s a smaller decision I’d defend just as strongly, and it surprised me a bit when I got into it: the worked examples the engine uses internally to calibrate its own outputs can never be built from real customer situations. Every one of those examples is fabricated: a fictitious company, fictitious technical details, mapped to a real framework clause, constructed carefully enough to mirror the difficulty and style of a genuine case without being traceable to any actual organization or person. It’s a small, almost invisible design decision, and it’s one of the ones I’d defend most strongly if a customer ever asked me directly how their data is handled during generation. The honest answer is that it isn’t handled during generation in any form that could identify them, because the system was built so that path doesn’t exist in the first place.

The Verdict

Does it solve the problem? Mostly, yes. A reviewer isn’t reading twenty identical-looking outputs and hoping to catch the one that’s wrong; they’re pointed, structurally, at the ones that need their judgment most. And the honest answer to “does my data touch this process” is no, because the system was built so that path doesn’t exist, not because someone remembered to redact something correctly on a given day.

Where I’m less confident about: the loop that’s supposed to learn from a reviewer’s edits over time and feed that back into future confidence levels only got partially designed in this stretch of work. I know roughly what shape it needs to take, abstracted patterns from edits rather than raw copies of what changed, with strict limits on how long even that abstracted version sticks around, but it isn’t locked yet, and I don’t want to overstate where that piece stands.

There’s also a genuine tension I haven’t fully resolved between speed and thoroughness. The two-stage approach, a quick check before the heavier generation step, exists specifically so the system doesn’t spend expensive effort writing a full clause for a case that isn’t ready yet. That’s the right instinct. But every stage boundary is also a place where something can be misclassified, and I’ll be watching closely once this is live for cases where the quick check gets it wrong in either direction, in both the direction of being too cautious and the direction of being too confident.

What I’m confident about is the core shape of the decision: generation isn’t one function that always behaves the same way, and it never touches personal data in the process of doing its job. Those two constraints together are what let me tell a customer, honestly, that the time it takes to get a draft back didn’t come at the cost of either their trust in the answer or their trust in how their information was handled to get there.

( See: PolicyForge — Why ranked results shouldn’t be a black box )

Your Turn

If you’ve used an AI tool that gave you one confident-sounding answer that turned out to be wrong, what would have changed your reaction if it had told you upfront how confident it actually was?