Stopping an AI from inventing facts about a person

Network Growth Assistant helps write short messages to real people: a first note, a thank-you, a check-in. A message that says “great talking to you last week” to someone I have never spoken to is worse than no message. So the app has one rule: a message may only say things the app can back up.

Most messages never come from an AI at all

The follow-up messages after someone accepts an invitation come from fixed templates, filled in with what the record holds: the person’s name and, if known, their school and field. Nothing in that step calls an AI service, so a draft cannot contain an invented fact. Where an AI does write text, its output goes through the same check before it is used.

What counts as a lie?

I wrote down the kinds of statements the app must never make, and turned each into a rule with a reason:

• Pretending we have met: “we spoke”, “as we discussed”, “remember me”.
• Claiming a mutual connection or a “2nd-degree” link.
• Asking for a referral or a job in a message.
• Including a link or a handle.
• Sounding promotional, such as “amazing” or a string of exclamation marks.

  { pattern: /\bmutual (connection|friend|contact)s?\b|\b(2nd|3rd|second|third)[- ]degree\b|\bwe have .* in common\b/i, reason: "claims a mutual connection or degree" },
  { pattern: /\b(refer(ral)?|refer me|hire me|send (me )?a job)\b/i, reason: "asks for a referral or job" },
  { pattern: /https?:\/\/|www\.|@\w/i, reason: "contains a link or handle" },

Each rule carries a reason, so a refused message comes back with the reason it was refused, such as “claims a mutual connection or degree”, not just “invalid”.

The harder rule: only names I was told

Banned phrases catch what I thought of. The second rule catches what I did not. Any capitalized word in the message that is not at the start of a sentence, and is not an everyday word, has to appear in the facts the app was given about that person. If a message mentions “Amazon” and Amazon is not in the record, it is refused.

  const knownText = ` ${known.filter(Boolean).join(" ").toLowerCase()} `;
  const words = t.split(/\s+/);
  for (let i = 0; i < words.length; i++) {
    const w = words[i].replace(/^[^\p{L}\d]+|[^\p{L}\d'’]+$/gu, "");
    if (!w || !/^\p{Lu}/u.test(w)) continue;
    const sentenceStart = i === 0 || /[.!?:—–-]$/.test(words[i - 1]) || /^["“(]/.test(words[i]);
    const lower = w.toLowerCase().replace(/[’']s$/, "");
    if (sentenceStart || COMMON.has(lower)) continue;
    if (!knownText.includes(lower)) return { ok: false, reason: `mentions “${w}”, which isn't in the evidence` };
  }
  return { ok: true };

Reading it in plain words:

• The facts are joined into one lowercase string to search in.
• Each word that starts with a capital letter is a candidate name. Words at the start of a sentence are skipped, because every sentence starts with a capital.
• A short list of everyday words, such as “hello”, “thanks” and “university”, is also skipped.
• Any other capitalized word must appear in the facts, otherwise the message is refused and says which word was the problem.

It also checks length, numbers, and your edits

The same function refuses a message that is empty or longer than its limit. Follow-up drafts get an extra rule against implying an earlier conversation, and a check on numbers. And because you can edit a draft before copying it, the check runs again on your edited version, so an edit cannot sneak in a claim the record does not support.

What this does not do

It is a filter, not a fact-checker. It cannot tell whether a true-sounding sentence is true, and the name check only looks at capitalized words, so a wrong claim written in lowercase would get past it. What keeps the risk small is the design around it: use a template wherever a template is enough, and refuse anything that cannot be traced back to the record.