OOF-AII-768876
Genderify’s AI to Predict a Person’s Gender Revealed by Free API Users to Exhibit Bias
A company's AI predicting a person's gender based on their name, email address, or username was reported by its users to show biased and inaccurate results.
- Event date
- 2020-07-28
- Country
- United States of America
- Severity
- Low
- Evidence state
- Single-source
Assessment status
Generated from available public incident evidence using OOF® governance methodology. It is not an official investigation, legal determination, regulatory finding, certification, or OOF® Approved Assessment.
View source record from AI Incident Database (Responsible AI Collaborative)
Assessment confidence dimensions
- Source corroboration
- Single-source
- Extraction completeness
- High
- Methodological assignment confidence
- Limited
Measures source-group corroboration only; it does not establish factual truth.
Measures populated source fields after deterministic normalization; it does not measure correctness.
Measures automated assignment support; substantive correctness requires authorized human review.
Known facts from the incident record
- The incident record identifies the event date as 2020-07-28.
- The incident record identifies the location as United States of America.
Reported claims
- A company's AI predicting a person's gender based on their name, email address, or username was reported by its users to show biased and inaccurate results.
Material unknowns
- Independent corroboration is not established in the available incident record.
Primary governance problem
The incident appears to expose a governance problem involving communication, information. Available reporting supports preliminary methodological interpretation, not a finding of fault or non-compliance.
Architecture intelligence
PGA™ — Prediction Governance Architecture
Primary architecture
The reported communication, information concern falls within the governed questions represented by PGA™ — Prediction Governance Architecture.
Governance focus: How should communication, information be governed in the conditions described by the available incident evidence?
Deeper methodology is withheld because the available evidence does not justify a Parent Standard or Core Module.
Preliminary governance questions
- How should communication, information be governed in the conditions described by the available incident evidence?
- What operational authority and controls applied to Genderify at the relevant time?
- What evidence would establish how yes produced or enabled the reported outcome?
- What additional independent evidence is required to resolve the material evidence gaps?
Evidence gaps and limitations
- Only one independent source group is represented; corroboration is not established.
- AI technology is not specified in the source record.