Branching/Snapshot failures

Incident Report for Platform.sh

Postmortem

On January 16th an orchestration layer software upgrade introduced a bug causing cloned branches being deployed without a copy of the data. The upgrade affected the following regions: AU, CA-1, EU-2, FR-1, DE-1, and US-2. We re-released a version of the software containing a fix for the identified bug on January 18th at 03:47 UTC, restoring the functionality of the affected service back to normal.

Posted Jan 22, 2019 - 17:14 UTC

Resolved

A patch to resolve this issue has been deployed and all services have returned to normal. For any further issues please contact us via the support desk https://support.platform.sh/

Posted Jan 18, 2019 - 04:36 UTC

Update

We are continuing to monitor for any further issues.

Posted Jan 18, 2019 - 04:23 UTC

Monitoring

A fix has been implemented and we are monitoring the results.

Posted Jan 18, 2019 - 04:22 UTC

Identified

We have identified the root cause of the cloning failures. Our engineers are currently implementing a fix.

Posted Jan 18, 2019 - 02:32 UTC

Investigating

We are investigating reports of branching and snapshots operations silently failing. We recommend avoiding these operations until we have investigated the problem.

Posted Jan 18, 2019 - 01:43 UTC

This incident affected: Australia (au.platform.sh), Canada (ca-1.platform.sh), Europe (France) (fr-1.platform.sh), Europe (West 2) (eu-2.platform.sh), and USA-2 (East 2) (us-2.platform.sh).