OS X builds are queueing
Incident Report for CircleCI
Postmortem

At 1429 UTC our OS X data center vendor experienced a network outage which caused our OS X fleet to stop taking jobs. This reduced our OS X fleet capacity tremendously and we immediately started to get a backlog of jobs.

At 1448 UTC the network was restored and we immediately started to drain the job backlog and we cleared the backlog at 1530 UTC.

We considered the incident as resolved at 1649 UTC.

Posted Mar 07, 2017 - 20:06 UTC

Resolved
This incident has been resolved.
Posted Mar 07, 2017 - 16:49 UTC
Monitoring
The queue is now empty and builds are being dispatched correctly. We're continuing to monitor the situation.
Posted Mar 07, 2017 - 16:12 UTC
Identified
Builds are being dispatched again, but we're still investigating the underlying cause
Posted Mar 07, 2017 - 15:30 UTC
Investigating
OS X builds are queueing. We are investigating the reason for that and will update in 30 mins.
Posted Mar 07, 2017 - 14:55 UTC