Temporary AWS Keys Appeared In Artifact Logs
Incident Report for CircleCI
Postmortem

Summary: AWS credentials that are valid for 30 minutes were revealed in build output during the validity window for users who already had access to the build.

Potential Impact: Any user with access to the build output had the potential to overwrite build artifacts for up to 30 mins after a build ran by using the same token combinations recorded in the build page.

What Happened: At approximately 3PM UTC on April 19, 2017, a customer informed the CircleCI Security Team that they saw AWS credentials in the upload logs for build artifacts. A review of code showed that these credentials were, as expected, expiring tokens for S3's "pre-signed URLs" that are scoped by bucket, key, and method along with being limited to a 30-minute lifespan. However, the command to upload the artifacts included the --verbose flag, causing the full request headers to be logged, including these tokens.

What We Did About It:

1) Engineers upgraded our code and Site Reliability Engineers rolled the change out across the full SaaS build environment.

2) CircleCI released version 1.47.2 of CircleCI Enterprise, which patches the issue. We recommend customers upgrade immediately.

Posted Apr 21, 2017 - 17:48 UTC

Resolved
AWS credentials that are valid for 30 minutes were revealed in build output during the validity window for users who already had access to the build.
Posted Apr 21, 2017 - 17:42 UTC