Deployments to PE clusters may break the site
Incident Report for Platform.sh
Postmortem

On March 21st at 11:30 UTC, Platform.sh deployed a new software version that added Unicode support to deploy and post-deploy hooks. The update process failed to properly handle empty hooks, causing the deploy procedure to exit during its execution. As a result, some of our Enterprise customers who were deploying at that moment, experienced a variable downtime of 10 to 75 minutes. Manual fixes were applied as needed in order to restore the functionality of these websites and a final version of the update was deployed at 15:53 UTC across all Enterprise clusters.

Posted Mar 27, 2019 - 18:51 UTC

Resolved
This incident is now resolved..
Posted Mar 21, 2019 - 16:36 UTC
Update
The second fix has now been deployed.
We are monitoring the systems and you can start pushing changes
Posted Mar 21, 2019 - 15:57 UTC
Update
The deployment of the second patch is continuing.
Posted Mar 21, 2019 - 15:34 UTC
Update
The cause of the inconsistencies has been identified, and a fix has been written: the fix is now being deployed to all the Enterprise clusters.
Posted Mar 21, 2019 - 14:42 UTC
Update
During the monitoring phase we have noticed inconsistencies. Please refrain from deploying while we investigate.
Posted Mar 21, 2019 - 14:10 UTC
Monitoring
The fix has now been deployed. We are monitoring the systems.
You can start pushing changes.
Posted Mar 21, 2019 - 12:56 UTC
Update
The deployment of the fix is still ongoing.
Posted Mar 21, 2019 - 12:35 UTC
Update
We are now deploying a fix to all Enterprise clusters.
Posted Mar 21, 2019 - 12:04 UTC
Identified
We have identified the issue and are working on a fix.
Posted Mar 21, 2019 - 11:37 UTC
Update
We are continuing to investigate this issue.
Posted Mar 21, 2019 - 11:31 UTC
Investigating
After an update in our deployment software layer, the deployments to PE clusters may break the site.
We recommend not to deploy new changes at the moment.
Posted Mar 21, 2019 - 11:30 UTC
This incident affected: Dedicated Enterprise.