Description
From approximately 23.44 UTC to 23.54 UTC on 2025-11-02, users were unable to log in to the European production instance. It is possible that users faced sporadic issues logging in also between approximately 23.00 UTC and 23.44 UTC. No other instances were affected.
Cause
The cause of the issue was an unexpected error in the identity provider (idp), leading to an accumulating memory consumption during the evening (UTC) of 2025-11-02.
From approximately 23.00 UTC users would face sporadic failures when trying to log in.
From approximately 23.44 UTC users could not log in.
At 23.54 UTC on 2025-11-02 the idp was restarted, and users could log in as expected again.
Corrective Action
Alerts were triggered when the idp did not respond and the idp was restarted again at 2354 UTC on 2025-11-02. After the restart users could log in as expected again.
Preventive Action
Preliminary investigation identified a few improvements that could be made to make the idp more resilient. Implementation of these improvements have been initiated while the investigation of the issue continues.