CLI, API, Console UI and SSH access experiencing degraded performance on DE 2

Incident Report for Platform.sh

Postmortem

What happened

An API-endpoint coordinator ended up in a degraded state after an upgrade was performed on the region.

Customer impact

API, CLI, Console, and SSH access were degraded for projects on the DE-2 region for roughly two hours. During this time, requests to the project API would intermittently respond with 502s. Traffic to live sites was not impacted throughout the incident duration.

What was done to resolve the incident

The degraded API-endpoint coordinator was recovered.

Posted Nov 21, 2023 - 22:42 UTC

Resolved

This incident has been resolved.
Posted Nov 21, 2023 - 22:39 UTC

Monitoring

A fix has been implemented and we are monitoring the results.
Posted Nov 21, 2023 - 21:48 UTC

Identified

The issue has been identified and a fix is being implemented.
Posted Nov 21, 2023 - 21:26 UTC

Investigating

We have detected degraded performance impacting CLI, Console UI, and SSH access on DE-2 projects. We are actively investigating, and working to restore service.

Please note that traffic to live sites is not impacted. Production sites are fully online and serving requests.

We will update you as soon as we have further information.
Posted Nov 21, 2023 - 21:11 UTC
This incident affected: Europe (Germany) (de-2.platform.sh).