OOF-AII-269442

High-Toxicity Assessed on Text Involving Women and Minority Groups

Google's Perspective API, which assigns a toxicity score to online text, seems to award higher toxicity scores to content involving non-white, male, Christian, heterosexual phrases.

Event date
2017-02-27
Country
United States of America
Severity
Low
Evidence state
Single-source

Assessment status

Generated from available public incident evidence using OOF® governance methodology. It is not an official investigation, legal determination, regulatory finding, certification, or OOF® Approved Assessment.

View source record from AI Incident Database (Responsible AI Collaborative)

Assessment confidence dimensions

Source corroboration
Single-source

Measures source-group corroboration only; it does not establish factual truth.

Extraction completeness
High

Measures populated source fields after deterministic normalization; it does not measure correctness.

Methodological assignment confidence
Moderate

Measures automated assignment support; substantive correctness requires authorized human review.

Known facts from the incident record

  • The incident record identifies the event date as 2017-02-27.
  • The incident record identifies the location as United States of America.

Reported claims

  • Google's Perspective API, which assigns a toxicity score to online text, seems to award higher toxicity scores to content involving non-white, male, Christian, heterosexual phrases.

Material unknowns

  • Independent corroboration is not established in the available incident record.

Primary governance problem

The incident appears to expose a governance problem involving assessed, communication, information. Available reporting supports preliminary methodological interpretation, not a finding of fault or non-compliance.

Architecture intelligence

PGA™ — Prediction Governance Architecture

Primary architecture

The reported assessed, communication, information concern falls within the governed questions represented by PGA™ — Prediction Governance Architecture.

Governance focus: How should assessed, communication, information be governed in the conditions described by the available incident evidence?

Deeper methodology is withheld because the available evidence does not justify a Parent Standard or Core Module.

Preliminary governance questions

  • How should assessed, communication, information be governed in the conditions described by the available incident evidence?
  • What operational authority and controls applied to Google at the relevant time?
  • What evidence would establish how Google Perspective Api produced or enabled the reported outcome?
  • What additional independent evidence is required to resolve the material evidence gaps?

Evidence gaps and limitations

  • Only one independent source group is represented; corroboration is not established.

Return to the public incident index