Viedoc 3 Japan outage
Incident Report for Viedoc
Postmortem

Report author

Jens Pettersson
Technical Fellow
Viedoc Technologies AB

Report issued

2023-APR-03

Description

System unavailable

Instances affected

Viedoc version 3 - Japan instance

Duration

2023-APR-01 03.02 -- 05.58 JST

Cause

Parts of the Viedoc 3 Japan instance still rely on co-location using Equinix services in Tokyo, Japan. This datacenter experienced complete interconnectivity blackout due to a hardware failure. See provider report at https://status.equinix.com/incidents/dc2wztpl1vc1. Please note that provider report starts 36 minutes after outage started.

Impact

Availability only. Data integrity and data confidentiality not affected.

Correction

Upstream internetwork provider replaced failed hardware at 05.58 JST.

Corrective action

This was an SLA breach from provider as network redundancy is guaranteed according to contract and installation of redundant internet connectivity (doubled pipes). Due to Preventive action described below, no further corrective actions will be taken at this point.

Preventive action

The provider will be decommissioned later this year and replaced in full with Microsoft Azure Japan IaaS.

Posted Apr 03, 2023 - 15:26 UTC

Resolved
This incident has been resolved.
Posted Mar 31, 2023 - 21:07 UTC
Update
Upstream provider technicians still sourcing new hardware.
Posted Mar 31, 2023 - 20:19 UTC
Update
Upstream provider technicians are sourcing new hardware.
Posted Mar 31, 2023 - 19:03 UTC
Identified
Our upstream provider is experiencing network outages
Posted Mar 31, 2023 - 18:46 UTC
This incident affected: Viedoc 3 - Japan.