OpenAI Software / SaaS North America

SafetyKit

Fraud Detection · Risk Management · Security and Compliance

SafetyKit needed to review huge volumes of risky multimodal content for fraud, prohibited activity, and compliance violations with higher nuance and lower cost than legacy moderation systems or manual review could provide.

10+ point performance gains on hardest vision tasks with GPT-5
16B tokens processed daily, up from 200M six months ago
95%+ accuracy reviewing 100% of customer content based on SafetyKit evals

Reported impact

Business outcome

SafetyKit expanded into more risk domains, increased moderation coverage and consistency, protected human reviewers from harmful content, and turned new OpenAI model releases into an operational edge for customers fighting rapidly evolving fraud and abuse.

  • Productivity Improvement
  • Risk Reduction
  • Scalability

Solution

SafetyKit built a portfolio of risk agents on top of OpenAI's reasoning and multimodal stack so each violation type can be handled by the model best suited to it. GPT-5 performs high-stakes multimodal reasoning across text, images, and user interfaces; GPT-4.1 follows detailed content-policy instructions in high-volume moderation flows; reinforcement fine-tuning pushes recall and precision beyond base-model performance; deep research adds real-time online investigation for merchants and verifications; and the Computer Using Agent automates policy tasks once handled manually. The platform routes each piece of customer content to the right agent for scams, policy disclosure, illegal products, or other risk categories, covering gray areas like embedded phone numbers, regional rules, and ambiguous product claims. SafetyKit also benchmarks every new model release against its hardest cases and can deploy winning models quickly, turning OpenAI model progress into immediate product gains.

Data flow

Customer content and signals are routed by risk category into specialized agents. The selected OpenAI model evaluates text, images, transactions, or listings against policy context, may run research or browser tasks, and returns structured decisions that SafetyKit can enforce or escalate.

Solution architecture

5 components · 4 layers
  1. Compute
    • GPT-5 Handles multimodal reasoning for subtle and high-stakes safety and compliance decisions.
    • GPT-4.1 Executes high-volume moderation tasks with strong instruction following.
  2. Serving
    • deep research Brings real-time online investigation into merchant-review and verification workflows.
  3. Orchestration
    • Computer Using Agent Automates complex policy tasks that would otherwise require manual review.
  4. Governance
    • Reinforcement Fine-Tuning Improves recall, precision, and policy consistency using SafetyKit's feedback data.

Architecture clues

  • Deep research and CUA extend the moderation system beyond static model inference
  • Each risk category is handled by a purpose-built agent
  • New OpenAI releases are benchmarked against SafetyKit's hardest cases and deployed quickly
  • RFT uses policy-specific analyst feedback to improve precision and recall

Evidence from the source

"16B tokens processed daily, up from 200M six months ago"
"GPT‑5 followed, and within days, it was deployed across their most demanding agents"
"Recent breakthroughs in model reasoning and multimodal understanding now make this more effective"
"SafetyKit’s agents leverage GPT‑5, GPT‑4.1, deep research, and Computer Using Agent (CUA) to review 100% of customer content with over 95% accuracy"
Transparency Evidence & methodology 90/100 reference value · 89% confidence · 1 source

Review how this reference was scored and trace each normalized claim back to its documented source.

01

Scoring methodology

How scores are calculated

Customer Reference Value measures usefulness as a customer reference. Evidence Confidence measures the quality and traceability of available evidence, not independent verification.

Customer reference value90/100
  • Problem clarity20/20
    Measures whether the customer's operational problem is specific enough to compare.
  • Implementation depth25/25
    Rewards named components, data flow, integrations, and architecture detail.
  • Outcome specificity20/20
    Measures whether the reported benefit is concrete rather than a generic claim.
  • Quantified impact15/25
    1 metric describe measurable impact.
  • Deployment scale10/10
    Rewards evidence of a live or scaled deployment and meaningful integration scope.
Evidence confidence89%
  • Source authority20/25
    The main evidence is an official vendor-published customer story.
  • Claim traceability21/25
    10 of 12 key fields link to a named source.
  • Direct evidence20/20
    4 direct source quotes retained.
  • Date provenance10/10
    The publication date comes from the source page or publisher.
  • Source transparency8/10
    1 source reference document publisher, URL, authority, retrieval, and supported fields.
  • Implementation specificity10/10
    Rewards source-backed components, data flow, quantified impact, and a specific outcome.
02

Traceability

Sources & provenance

Field coverage shows which facts each source supports. Dates are shown only when documented by the source.

  1. OpenAI

    SafetyKit customer story

    First-party
    https://openai.com/index/safetykit/
    Source type
    Vendor-published customer story
    Published
    Sep 9, 2025
    Supports
    • Customer
    • Business problem
    • Solution
    • Products
    • Solution components
    • Data flow
    • Integrations
    • technical area
    • use case category
    • Business outcome
    • outcome category
    • Metrics
    • Architecture
    • evidence quotes
    • Publication date

Date evidence

  • Published on source page Sep 9, 2025 · OpenAI · First-party
    Open date source
Not separately sourced Industry, Region, Company size. Treat these normalized attributes as lower-confidence until a dedicated source is added.