Degraded Performance on Select Virtual Machines

Incident Report for TCF Ventures

Resolved

This incident has been resolved.
Posted May 15, 2025 - 22:00 EDT

Update

The RAID-backed storage volume on EU1 has been successfully reassembled and remounted following a post-reboot mount delay. Impacted virtual machines have been brought back online, and service has been restored.

We are currently monitoring the environment to ensure continued stability before marking this incident as resolved.
Posted May 15, 2025 - 18:59 EDT

Identified

Following a routine system reboot, one of our hypervisor nodes encountered a delay in remounting a RAID-backed storage volume. As a result, a subset of virtual machines experienced temporary unavailability. The volume has since been successfully reattached, and restoration efforts are actively underway.

Our engineering team is proceeding with the controlled recovery of affected instances and will continue to monitor system performance to ensure stability throughout the process.
Posted May 15, 2025 - 18:46 EDT

Investigating

We’re aware of an issue impacting a limited number of virtual machines on one hypervisor. Affected systems may be temporarily unavailable or exhibit degraded performance. Our infrastructure team is actively investigating and working to restore full service. We’ll provide updates as they become available.
Posted May 15, 2025 - 18:34 EDT
This incident affected: Servers (EU.1) and Hosting Services (Freyja, Fenrir).