What to Do If It Worked
An attack through prompt override shows in the chat's behavior. It does something other than what it was asked.
The signs
Be on alert if you notice any of this:
- the chat offers to open a link you never asked about;
- an image or an address with a long string of characters appears in the answer;
- the chat reports that it sent an email or created a record, though you asked it to analyze a document;
- the analysis of a document keeps praising one option against what you read yourself;
- the chat replies that it cannot tell you about part of its own work.
That last phrasing shows up in hidden instructions especially often.
First steps
Stop the conversation. Do not send new messages into that chat and do not ask for explanations: the instruction is still in the thread and will keep being carried out.
Disconnect the connectors if you had connected any. That closes access to your mail and files immediately, without waiting for an investigation.
Tell your security team or your manager. Say which document or email you were working with and what the chat did.
What not to do
Do not run the attack again "to be sure." A repeat carries the instruction out one more time.
Do not forward the infected document to colleagues with a note saying "look at this." Their chat will read the same thing.
Do not treat what happened as your fault. These attacks are built for ordinary workday inattention, and in the two cases described here nothing at all was required of the user.