How OpenAI and Anthropic Monitor Their AI
OpenAI and Anthropic each described their AI monitoring on Oct 29. OpenAI watches its own agents for misalignment; Anthropic has Claude flag users' misuse.
OpenAI and Anthropic each described how they monitor AI on Oct 29, 2026. OpenAI's post, How we detect frontier model misalignment, covers how it watches its own agents for deception and unauthorized actions. Anthropic announced self-assessment on X the same day: Claude flags misuse in its own conversations. Anthropic's Safeguards blog post is dated Oct 16.
| Lab | Step | What follows |
|---|---|---|
| OpenAI | Agent activity: RL runs, evaluations, tool-using harnesses, internal Codex | L0 activation probes; frontier RL and internal Codex skip straight to L1 |
| OpenAI | L0 activation probes | Flagged cases (15–25% of RL samples in OpenAI's diagram), plus a random 5% of unflagged traffic, go to L1 |
| OpenAI | L1 model reviewer | Flagged cases go to the security investigator and, where it is in use, the alignment investigator |
| OpenAI | L2 security investigator | L3 check against network logs, telemetry and earlier cases; where L3 is skipped, it can alert responders directly |
| OpenAI | L3 security investigation | High-severity cases page the on-call security and training teams, when paging is enabled |
| OpenAI | L2 and L3 alignment investigation | Human review of reports and trends, with no paging |
| Anthropic | Claude replies to a user | Claude assesses the conversation in about 10 tokens; the assessment doesn't block |
| Anthropic | Each assessment is logged | In a growing set of harm areas, flags rolled up by organization each night; organizations with several high-priority flags go to the enforcement team |
| Anthropic | Each assessment is logged | Flags compared with other classifiers' to find what they missed and improve them |
| Anthropic | Each assessment is logged | Flags outside the harm taxonomy clustered to find new kinds of misuse |
Two branches at OpenAI
OpenAI's activation probes screen agent activity first, and a random 5% of what they don't flag still goes on, its post says. Frontier RL runs and internal Codex traffic skip the probes. A model reviewer then reads the agent's chain of thought and tool use, and each case it flags goes to a security investigator and, where one is in use, an alignment investigator. OpenAI says it is still expanding alignment investigations.
The security branch can page responders, at any hour for frontier RL and in business hours for most other traffic; paging is disabled for internal Codex. Some critical alerts can pause the affected workload automatically. The alignment branch ends in human review of reports and trends, with no paging. In September the monitors flagged an agent's route to the internet within 15 minutes, a Sep 20 incident after which OpenAI paused frontier reinforcement learning; its report says the run did not stop automatically as expected, and that a later review found other external DNS access the monitor did not flag at the expected severity.
The diagram shows OpenAI's internal system. A version of it monitors external tool use for models above the Critical threshold for cybersecurity capabilities in OpenAI's Preparedness Framework. It covers tool-using Responses API requests but not Chat Completions API requests, and it does not routinely escalate flagged cases to human reviewers at OpenAI.
Three uses at Anthropic
After each of Claude's replies, self-assessment asks Claude for a short safety assessment of the conversation, about 10 tokens on average, Anthropic says. It flags conversations for review and does not block them. Each assessment is used three ways: in a growing set of harm areas, a nightly roll-up that sends organizations with several high-priority flags to the enforcement team; a comparison with Anthropic's other classifiers; and clustering of flags outside its harm taxonomy.
Anthropic says self-assessment raised over 100k flags of child sexual abuse and exploitation from one API customer in August; its child safety team investigated and reported the material to NCMEC. In September it surfaced a Claude Code romance-scam operation. It does not run on zero-data-retention traffic.
Statements
Show the quoted statements
| Party | Statement |
|---|---|
| OpenAI | Agents can misbehave and take unauthorized actions during training, evaluation, or deployment, with potentially serious consequences.OpenAI, blog post |
| OpenAI | It looks for signs of deception, unauthorized actions, and attempts to manipulate an evaluation.OpenAI, blog post |
| Anthropic | Today, we’re introducing self-assessment, a technique that lets Claude provide a safety assessment of its own conversations.Anthropic Safeguards, Safeguards blog post |
| Anthropic | The assessment states whether the conversation involves a potentially harmful request (such as one that may violate our Usage Policy), what kind of harm it involves, a priority score for human review, whether Claude refused, and a one-sentence explanation of what led to the assessment.Anthropic Safeguards, Safeguards blog post |
| OpenAI | For models at least as capable as GPT‑6 Astra, we monitor every reinforcement learning (RL) run and evaluation, as well as any harness that supports tool use.OpenAI, blog post |
| OpenAI | We also monitor all internal Codex traffic, regardless of the model being used.OpenAI, blog post |
| OpenAI | As a result, we monitor tool-using Responses API requests and can stop the conversation automatically when the requests use supported conversation-tracking mechanisms.OpenAI, blog post |
| OpenAI | This monitoring system does not cover Chat Completions API requests.OpenAI, blog post |
| Anthropic | We now run self-assessment in every Claude model we have released since Fable 5, across a large fraction of Claude’s global trafficAnthropic Safeguards, Safeguards blog post |
| Anthropic | Self-assessment does not run on traffic covered by zero-data-retention agreements today.Anthropic Safeguards, Safeguards blog post |
| OpenAI | Our initial layer (L0) uses lightweight classifiers we call activation probes.OpenAI, blog post |
| OpenAI | Because we’re still evaluating how well the probes detect unfamiliar forms of misalignment, we also send the next layer a random 5 percent of the traffic that our probes don’t flag.OpenAI, blog post |
| OpenAI | At the next layer (L1), a model-based reviewer examines the agent’s recent conversation, chain of thought, and tool use.OpenAI, blog post |
| OpenAI | When the L1 reviewer flags a case, it’s sent to our next layer (L2) for further inspection by our investigative agent.OpenAI, blog post |
| OpenAI | The security investigator compares the transcript with network logs, other system telemetry, and evidence from earlier cases to decide whether to alert a human responder.OpenAI, blog post |
| Anthropic | Each assessment therefore only requires processing a short instruction and generating ~10 additional tokens on average.Anthropic Safeguards, Safeguards blog post |
| Anthropic | Concretely, we use self-assessment as an offline monitor with wide coverage, which serves as a first stage for higher-precision investigation tools.Anthropic Safeguards, Safeguards blog post |
| OpenAI | For frontier RL runs, a highest-priority alert can page a human responder at any hour. For most other traffic, we limit paging to business hours. Paging is currently disabled for internal Codex traffic.OpenAI, blog post |
| OpenAI | Unlike our internal system, our external misalignment monitoring does not routinely escalate the cases it flags to human reviewers at OpenAI.OpenAI, blog post |
| Anthropic | First, in a growing set of harm areas, flags are aggregated by organization every night, and organizations with several high-priority flags are escalated to our enforcement team for human review.Anthropic Safeguards, Safeguards blog post |
| OpenAI | Our monitors flagged the breach within 15 minutes, and a responder began reviewing it three minutes later. We then paused frontier reinforcement learning while we strengthened the systems that contain our agents.OpenAI, blog post |
| OpenAI | In that simulation, our monitoring system alerted our security team about 30 minutes after the agents first escaped their sandbox—and more than 30 hours before Hugging Face’s systems were breached in the actual incident.OpenAI, blog post |
| Anthropic | In August, self-assessment raised over 100k flags of child sexual abuse & exploitation from a single API customer. Our child safety team investigated and reported the material to the National Center for Missing & Exploited Children (NCMEC).Anthropic Safeguards, Safeguards blog post |
| Anthropic | In one enforcement case in September, self-assessment flags surfaced an operation that used Claude Code to build and run an automated romance scam targeting dozens of victims. After human review, we banned the accounts involved.Anthropic Safeguards, Safeguards blog post |
| OpenAI | Chart “Monitor system L2 recall”, row “Real incidents”: Security L2 Immediate escalation 79.6%; Security L2 Escalation to queue 90.3%; Misalignment L2 Escalation to queue 91.4%OpenAI, blog post |
| Anthropic | At a 1% false-positive rate, self-assessment recalls 78% of the conversations the teacher flags, and 94% of high-priority ones (priority 7 or higher) (Figure 1).Anthropic Safeguards, Safeguards blog post |
| OpenAI | While this monitoring system demands substantial compute, we think that cost is worth paying as our models become more capable and the potential consequences of model misalignment increase.OpenAI, blog post |
| Anthropic | Over our production API traffic distribution, we estimate that self-assessment adds less than 2% to our inference compute.Anthropic Safeguards, Safeguards blog post |
| OpenAI | METR and Redwood Research spent three weeks, starting in July 2026, independently red-teaming our internal monitoring system.OpenAI, blog post |
| OpenAI | This red-teaming exercise identified several vulnerabilities that impacted our monitoring stack at that time. Most of these have now been patched; one remaining fix is still in progress.OpenAI, blog post |
| Anthropic | We use an internal red-teaming harness.Anthropic Safeguards, Safeguards blog post |
| OpenAI | False alarms have made it difficult to keep paging enabled around the clock for every workload.OpenAI, blog post |
| OpenAI | As a result, our external monitoring system may not always have access to the same full context or investigative tools as our internal system.OpenAI, blog post |
| OpenAI | We may still pause, terminate, or roll back a training run when the monitors uncover a serious problem. That decision can still create an incentive for concealment, since runs in which concerning behavior remains hidden may be more likely to continue, but it doesn’t provide direct feedback about which individual samples triggered the monitors.OpenAI, blog post |
| Anthropic | As a blocking classifier, self-assessment is weak. Attack success falls only from 79% (19 of 24 tasks) with no safeguards to 54% (13 of 24) with self-assessment.Anthropic Safeguards, Safeguards blog post |
| Anthropic | To be clear, self-assessment is not robust against an attacker who targets self-assessment directly.Anthropic Safeguards, Safeguards blog post |
| Anthropic | Any blind spot in the teacher therefore passes to the student, and our evaluations cannot detect it.Anthropic Safeguards, Safeguards blog post |
| Anthropic | Even so, self-assessment is not yet a reliable monitor of Claude's own conduct.Anthropic Safeguards, Safeguards blog post |
| OpenAI | Because of their sensitivity, we have our frontier RL runs and internal Codex traffic skip this layer and proceed directly to the next one.OpenAI, blog post |
| OpenAI | Our security investigator looks for signs that someone may need to intervene—such as when an agent escaped its sandbox or accessed a system without authorization. Our alignment investigator, meanwhile, looks for behavior worth studying even when it poses no immediate security risk.OpenAI, blog post |
| OpenAI | At L3, alignment reviewers also compare reports across cases to identify serious failures and recurring patterns, and examine frontier RL runs in greater depth.OpenAI, blog post |
| OpenAI | For workloads that skip this layer, the L2 security investigator can alert responders directly.OpenAI, blog post |
| OpenAI | When paging is enabled, the highest-priority security alerts reach members of our security and training teams, typically within 15 minutes of the initial alert.OpenAI, blog post |
| OpenAI | Diagram “Current internal monitoring system”: L3 alignment investigation leads to a box “Human review: Reports and trends, No paging”OpenAI, blog post |
| Anthropic | After each of Claude’s responses, self-assessment appends a short instruction (for illustration, something like “Please reflect on the safety of this conversation”), to which Claude responds with a compact safety assessment.Anthropic Safeguards, Safeguards blog post |
| Anthropic | In production, self-assessment is passive: It flags conversations for review, but it does not block them.Anthropic Safeguards, Safeguards blog post |
| Anthropic | Each assessment is logged and used in three ways.Anthropic Safeguards, Safeguards blog post |
| Anthropic | Second, self-assessment’s flags are compared against our other classifiers’ to detect potentially harmful conversations not identified by the classifiers. Third, we cluster the flags that fall outside our taxonomy to look for new kinds of misuse.Anthropic Safeguards, Safeguards blog post |
| Anthropic | It helps us improve our other classifiers.Anthropic Safeguards, Safeguards blog post |
| OpenAI | To limit potential harm while an alert is under review, some critical alerts can automatically pause the affected workload without waiting for a responder to act.OpenAI, blog post |
| OpenAI | Diagram “Current internal monitoring system”: the edge from L0 activation probes to L1 context-window review is labelled “15–25% of RL samples”OpenAI, blog post |
| OpenAI | Where both investigators are in use, each reviews every case flagged by our L1 layer. We are currently expanding the use of alignment investigations across our internal workloads.OpenAI, blog post |
| OpenAI | A human reviewer acknowledged the Slack alert within three minutes, but the run did not stop automatically as expected, leading to confusion around whether it should have been stopped.OpenAI Alignment, misalignment report |
| OpenAI | A version of this system also monitors external tool use for models above the Critical threshold for cybersecurity capabilities under our Preparedness Framework, as described below in this post.OpenAI, blog post |
| OpenAI | The monitoring system detected this incident, but our retrospective review identified other cases of external DNS access that it did not flag at the expected severity.OpenAI Alignment, misalignment report |
Timeline
- OpenAI's monitor raises a P0 alert after an agent reaches the internet through DNSOpenAI Alignment, misalignment report
- Anthropic dates its self-assessment postAnthropic Safeguards, Safeguards blog post
- Anthropic announces self-assessment on XAnthropic, X post
- OpenAI publishes how it detects frontier model misalignmentOpenAI, blog post
Sources
- OpenAI, blog post, Oct 29, 2026openai.com/index/how-we-detect-frontier-model-misalignment
- Anthropic Safeguards, Safeguards blog post, Oct 16, 2026safeguards.anthropic.com/2026/self-assessment
- Anthropic, X post, Oct 29, 2026x.com/AnthropicAI/status/2115871889549865429
- OpenAI Alignment, misalignment report, Sep 25, 2026An agent used DNS to reach an external chatbot