Rendering job backlog causing slow build times

Incident Report for Percy

Resolved

This incident has been resolved.
Posted Aug 14, 2019 - 04:27 PDT

Monitoring

A fix has been implemented and the backlog of jobs is draining. We will continue to monitor until the issue is fully resolved. We have also temporarily provisioned additional rendering resources to help work off the backlog of jobs more quickly for the customers affected. Thank you for your patience as we work to resolve this issue.
Posted Aug 13, 2019 - 23:44 PDT

Identified

We have identified an issue with an internal job processing system that is causing a backlog of rendering jobs, so some builds are not processing as quickly as expected. We are working on a fix and will update this when it is in place.
Posted Aug 13, 2019 - 23:14 PDT
This incident affected: Rendering infrastructure.