Increased Error Rates
Incident Report for WattTime API
Postmortem

After further investigation it was determined that an automated upgrade utility triggered when it was not expected, and failed. This then retried a number of times before timing out.

In order to stabilise the situation an emergency upgrade was conducted, using a zero-downtime approach. This upgrade was successfully completed and all endpoints are now stable.

Posted Jul 03, 2023 - 03:22 PDT

Resolved
An automated database migration event caused an increase in error rates on the API. This has been identified and resolved.
Posted Jul 03, 2023 - 01:13 PDT