Build System Interruption
Incident Report for CircleCI
Resolved
This incident has been resolved.
Posted 5 months ago. Mar 24, 2018 - 02:48 UTC
Monitoring
We have finished re-following all users
Posted 5 months ago. Mar 24, 2018 - 02:26 UTC
Update
We are continuing to work on re-following all users to their previously followed projects.
Posted 5 months ago. Mar 24, 2018 - 01:38 UTC
Update
We are continuing to work on re-following all users to their previously followed projects.
Posted 5 months ago. Mar 24, 2018 - 00:59 UTC
Update
We are continuing to process users and re-follow them to their previously followed projects.
Posted 5 months ago. Mar 24, 2018 - 00:14 UTC
Update
We are continuing to work on re-following all users to their previously followed projects.
Posted 5 months ago. Mar 23, 2018 - 23:45 UTC
Update
We are continuing to work on re-following all users to their previously followed projects.
Posted 5 months ago. Mar 23, 2018 - 23:09 UTC
Update
We have re-followed some users to their projects and are continuing to work on re-following all users to their previously followed projects.
Posted 5 months ago. Mar 23, 2018 - 22:40 UTC
Update
We are continuing to re-follow users to projects. If you continue to experience issues with projects not building or being unable to follow a project, please follow the instructions in the previous update to reset your project webhooks.
Posted 5 months ago. Mar 23, 2018 - 21:38 UTC
Update
We are currently working on re-following users to projects that they were previously following. If we have not gotten to your project yet you can also repair this issue manually by having your organization's github administrator do the following steps:

1. Remove all CircleCI webhooks and services from your GitHub repo settings.
2. Tell the project to stop building on CircleCI through "Project settings".
3. Add the project back on CircleCI.
Posted 5 months ago. Mar 23, 2018 - 20:43 UTC
Update
We are in the process of restoring project access and additionally investigating reports of project configurations not being used on new builds.
Posted 5 months ago. Mar 23, 2018 - 20:05 UTC
Identified
Our upstream provider has deployed a fix and we have confirmed it to work. We expect all GitHub users to regain access to their existing projects however we are currently correcting secondary issues that may be occurring due to the initial upstream outage.
Posted 5 months ago. Mar 23, 2018 - 19:37 UTC
Monitoring
Our upstream provider has deployed a fix and we are able to read access rights again. We are currently monitoring our build system and are investigating further steps on our end to correct any potential remaining permissions issues.
Posted 5 months ago. Mar 23, 2018 - 19:21 UTC
Identified
Our upstream provider has determined the root cause and is currently rolling out a fix for the issue causing GitHub users to have their project access revoked.
Posted 5 months ago. Mar 23, 2018 - 19:04 UTC
Update
We are currently working with our upstream providers to determine the issue preventing access to GitHub projects.
Posted 5 months ago. Mar 23, 2018 - 18:54 UTC
Update
We have temporarily disabled project access revocation. We continue to investigate the root cause of this problem and will provide more information as it becomes available.
Posted 5 months ago. Mar 23, 2018 - 18:36 UTC
Investigating
We are currently investigating an issue causing project access to be revoked, preventing builds from running. We will provide updates when we have further information.
Posted 5 months ago. Mar 23, 2018 - 18:13 UTC