Issues with captcha completion workflow
Incident Report for hCaptcha
Postmortem

Immediate cause: A transient database connectivity issue caused some data to be inaccessible to the sync process responsible for distributing it to edge nodes, causing a new constraint check within siteverify to fail for some users as this code change deployed across our network and edge services restarted without access to the required data.

Root cause: The root cause of this issue was an unintentional dependency on database access added during a recent code change to more stringently enforce some per-user constraints on the siteverify call, which has now been addressed.

Resolution: The database connectivity issue was resolved, and the constraint check was improved to handle the no-data-available case correctly.

Ongoing work: All parts of our system are designed to continue functioning fully even during internal issues like database connectivity failures. We are implementing an additional set of CI/CD safeguards to more aggressively verify that this behavioral contract is maintained for all code changes.

Posted Feb 10, 2020 - 20:19 UTC

Resolved
This incident has been resolved.
Posted Feb 10, 2020 - 19:54 UTC
Identified
The issue has been identified and a fix is being implemented.
Posted Feb 10, 2020 - 19:50 UTC
Investigating
Currently investigating an issue with siteverify calls.
Posted Feb 10, 2020 - 19:40 UTC
This incident affected: Captcha API.