Full outage on EU Region

Incident Report for Platform.sh

Postmortem

During an update on the edge layer to replace the development environments' TLS certificates, a kernel bug prevented the restart. The issue caused the whole EU region to go down on November 15, 2018 for about 15 minutes between 15:58 and 16:13 UTC. All sites hosted on the region were down during that time.

Engineers quickly manually rebooted the 3 gateways which brought the service back online. Kernel versions are in the progress of being updated on all regions to ensure that this does not recur.

Posted Nov 30, 2018 - 18:56 UTC

Resolved

1721 UTC This incident is now resolved. All services are operating nominally.

Posted Nov 15, 2018 - 17:21 UTC

Monitoring

1611 UTC The edge layer has been restored and all services should be back to normal.

Posted Nov 15, 2018 - 16:12 UTC

Update

1608 UTC During an update on the edge layer to replace the development environments' TLS certificates, a kernel bug was encountered that prevented an automated restart. We're working on bringing back the edge layer.

Posted Nov 15, 2018 - 16:08 UTC

Identified

1605 UTC Operations has identified the issue and is in process of restoring service.

Posted Nov 15, 2018 - 16:05 UTC

Investigating

We have identified an issue on the EU Region that is affecting all live sites, deployments and UI access. We are working to resolve this issue.

Posted Nov 15, 2018 - 16:04 UTC

This incident affected: Europe (West) (eu.platform.sh).