Resolved
This incident is resolved. The fix has been deployed to production and all affected pipelines have recovered and worked through the buffered backlog. Thanks for your patience.
Monitoring
We're rolling out a fix to production now and monitoring as affected pipelines come back online and work through the buffered backlog.
Identified
We've identified the cause of S3 output pipelines failing to start and are preparing a fix for release to production. In the meantime, incoming data is being safely buffered and will be processed once pipelines recover, so no data will be lost.
Investigating
We're investigating an issue causing pipelines configured with S3 outputs to fail to start in production. Pipelines with other output types are not affected. We've identified the cause and a fix is being rolled out.
