UI Inaccessible and Workflows not running
Incident Report for CircleCI
Postmortem

Monday, November 23, 2020 update on November 18 incidents

Beginning on the morning of November 18 and throughout the day our platform experienced a series of issues that impacted our ability to run customer workflows, and in some cases, made the UI unavailable. It is important to note that we do not believe any customer jobs were lost as a result of this issue. All jobs were eventually run, albeit with a significant delay in some instances.

We value our customers tremendously and apologize for any disruption you may have experienced in your work. We plan to publish a full incident report when we have completed our incident review later this week. We are confident in the root cause of these incidents. As a first step, we have made monitoring changes to ensure they do not happen again, and will share more about the additional steps we are taking to address these issues. Thank you for your support and patience.

Posted Nov 23, 2020 - 21:07 UTC

Resolved
This incident has been resolved. All systems are fully recovered and operational.
Posted Nov 18, 2020 - 15:27 UTC
Monitoring
We are currently monitoring this incident and our systems are recovering.
Posted Nov 18, 2020 - 15:10 UTC
Update
We are continuing to investigate this issue. We are observing intermittent issues in the UI while attempting to access Dashboard and project views. Next update will be in 20 minutes.
Posted Nov 18, 2020 - 14:46 UTC
Investigating
We are currently investigating an issue impacting workflows and the CircleCI UI.
Posted Nov 18, 2020 - 14:22 UTC
This incident affected: Pipelines & Workflows and CircleCI UI.