This incident is resolved. macOS builds are now processing normally. All previously impacted components are fully operational.
Posted Mar 10, 2020 - 20:38 UTC
Update
We are seeing queue times have recovered for Docker, Machine, and Workflows. For macOS, jobs utilizing resource_class large have recovered. macOS jobs utilizing resource_class medium will continue to experience long queue times while we work through a backlog.
Posted Mar 10, 2020 - 19:44 UTC
Monitoring
We have deployed a fix and are currently monitoring the results. We have scaled up our systems to work through the current backlog. There will be a longer recovery time for the macOS fleet at this time.
Posted Mar 10, 2020 - 18:46 UTC
Identified
We have identified the cause of long build queue times and we are currently implementing a fix.
Posted Mar 10, 2020 - 18:34 UTC
Update
We are continuing to investigate the cause of slow builds.
Posted Mar 10, 2020 - 18:32 UTC
Update
Our teams are continuing to investigate the issue causing builds to run slow. We will update our status page every 20 minutes until the incident has been resolved.
Posted Mar 10, 2020 - 18:08 UTC
Investigating
We are currently investigating an issue causing all jobs to queue and long build queue times.
Posted Mar 10, 2020 - 17:41 UTC
This incident affected: Docker Jobs, Machine Jobs, macOS Jobs, and Pipelines & Workflows.