Apple's reviewer is your AI app's first jailbreaker

My AI app got rejected by Apple, and the rejection pointed at the wrong thing. The notice cited a metadata field, the description and promotional text, as the problem. I spent two rounds dutifully rewording metadata. It was not the metadata.

What actually happened: the reviewer made a fresh account, picked one of the app's characters, and tried to make it say something it should not. It did. The cited field was just where Apple pinned the proof. The real problem was that my model could be talked into misbehaving, and a stranger whose entire job is to find that out found it in about three messages.

That experience rewired how I think about shipping anything with a model in it.

The reviewer is an adversarial user, and so is everyone else

When you ship a normal app, review checks that it does what you say. When you ship an AI app, review checks what it can be provoked into doing. Those are completely different bars. You are no longer responsible only for the content you wrote. You are responsible for the content your model can be steered into generating by someone actively trying.

Assume your first user is hostile. In review, they literally are. After launch, some of them will be too.

Fix the cause, not the citation

The most useful lesson was about debugging the rejection itself. Apple cites a field because they need to point at something concrete. That citation is evidence, not diagnosis. Reword the cited field, resubmit, and you will often get rejected again, because you treated the symptom.

So before changing a single word, I learned to reconstruct what the reviewer actually did: which account, which steps, which conversation. The rejection makes sense the moment you stop reading the cited field and start reading the session behind it. Find the action that produced the bad output and fix that. The metadata was never the problem. The model's behavior under pressure was.

Models default to giving in

Here is the uncomfortable thing about guardrails. A model wants to be helpful, and "helpful" shades into "compliant" under repetition. Refuse once, get asked again, and again, and the default behavior is to eventually cave. "Fine," it says, and then it does the thing. The politeness training works against you here.

So refusal cannot be a single soft line in the prompt. It has to be explicit, and it has to explicitly survive being asked repeatedly. The model needs to be told, in effect, that "no" does not become "yes" just because someone keeps pushing. If you do not say that, the model reads persistence as a signal to relent.

Do not put a quotable bad example in the prompt

When you write guardrails it is tempting to show the model exactly what not to produce, with a concrete example. Do not. I did once, with a vivid line. The model lifted that exact line into a real conversation, verbatim, because it was the nearest thing to reach for. The "do not say this" example became a script for saying it.

Describe the forbidden shape abstractly: a bracketed placeholder, a category, never a quotable sentence. The same rule that makes good examples dangerous makes bad examples worse. Models copy what is concrete.

Test the attack before they do

The fix that actually closed it was a small adversarial sim: a script that plays a user trying to jailbreak my own app, runs the nastiest flows, and checks the output. Two minutes, a few cents, run before every submit. If my own bot survives my own worst attempts, the reviewer's three messages are far less likely to land.

You would not ship a login form without trying a wrong password. Do not ship a model without trying to break it on purpose first.