Summary

Since October 1, CircleCI customers have experienced multiple incidents which have caused delays and failures in customer pipeline execution. There are two (unrelated) causes, both of which the CircleCI engineering team is actively mitigating:

• Available Cloud Computing Capacity
• Internal Infrastructure

Available Cloud Computing Capacity

• Problem: Demand for high-performance cloud instances is rising extremely rapidly across the industry, which puts pressure on the cloud provider instance types that we use to execute customer jobs.

• Incidents:

  • Elevated wait times for machine jobs

    • Increased task wait times for Docker Gen2
    • Elevated level of infra fails on customer jobs
    • Delay on starting Machine Job Tasks
    • Delays starting Gen 2 Docker Jobs

• Mitigations and Resolutions:

  • We are expanding our set of cloud computing regions to include additional regions with available high-performance instances.
  • We are also working to secure additional guaranteed capacity from our cloud computing partners in our existing regions.
  • We are further expanding the set of instance types we can offer to customers.
  • We will publish a detailed Incident Report on these incidents on October 9, 2026.

Internal Infrastructure

• Problem: We assign customer workflows over multiple compute providers via a workflow orchestration service. This service is currently gated by the write throughput of a single database.

• Incidents:

  • Delays in pipelines, workflows, UI data, and notifications
  • Emergency Infrastructure Maintenance

• Mitigations and Resolutions:

  • By October 23, 2026, we plan to implement multiple redundant workflow orchestration service instances, each with its own database, which will enable us split the orchestration load between them.

Our Commitment

For avoidance of doubt,

• These particular incidents are not related to ongoing outages at Github
• We are not currently migrating our infrastructure or services
• We are not currently under cyberattack

Customer experience is our top priority, and we commit to continually improving the reliability of our systems to match the trust that our customers place in us. We thank our customers for their patience while our team worked to resolve these incidents and works to prevent future incidents.

Please reach out to our support team with any questions or concerns.