[STO2] Storage issue in STO2 region
Incident Report for Cleura Public Cloud
Resolved
As we have 2 open incidents in parallell, we will close this one as the migration work is done.
Next update will be in the other incident post.
Posted Sep 16, 2021 - 09:36 CEST
Update
Dear customer,

We would like to inform you about the current status for the storage work in STO2.

* The process of populating the disks with data continues.
* Syncronization of volumes and VM images has started today (Wednesday, September 8th).
* Volume syncronization will take approximately a week (-> Monday, September 13th), and will be done continuously in the background.

Customer impact:

* Continuous improvements on the stability of our storage system as more and more data is offloaded onto the new storage.
* We do not expect the sync of data to cause any disruptions to your services.
Posted Sep 08, 2021 - 10:05 CEST
Update
Dear customer,

As you know, our STO2 region has had degraded performance during the last two months. We are resolving this issue by installing NVMe-based hardware. Here is an update on the current status.

* New disks are now installed and are production ready.
* The process of populating the disks with data has started 4/9.
* This process will take some time to complete, but we expect to get continuous improvements on the stability of our storage system as more and more data is offloaded onto the new storage.

We do not expect the sync of data to cause disruptions to your services.

If you have further questions, please reach out to the City Network support.
Kind regards,
City Network
Posted Sep 07, 2021 - 07:13 CEST
Update
We are still working with installation of the new storage nodes. This work will take a couple of days and including a test procedure before going into production.

We have also made some minor changes to and for the time being, these changes have improved the stability of the cluster.
Posted Aug 18, 2021 - 15:05 CEST
Update
Update:
Today Monday, we have continued the work to keep storage-related disruptions to a minimum.

In addition to monitoring and performing needed adjustments, we are also underway to rack-mount and prepare new storage nodes that will be added to the storage cluster.

The physical installation will be completed today, and the next step with software installation will continue tomorrow.

We will update the daily progress here on cnstatus.
Posted Aug 16, 2021 - 17:18 CEST
Update
We are continuing to monitor this incident closely and we expect to have more information to share regarding this incident on Monday.
Posted Aug 15, 2021 - 10:39 CEST
Monitoring
We are continuing with close monitoring of this incident of storage cluster. Meanwhile we are preparing next steps to permanent fix this issue.
Posted Aug 13, 2021 - 16:07 CEST
Update
Technicians are still working on this incident. We are currently working on decreasing the load on the storage cluster.
Posted Aug 13, 2021 - 10:30 CEST
Identified
The issue has been identified and a fix is being implemented.
Posted Aug 13, 2021 - 07:47 CEST
Investigating
We are currently investigating this issue.
Posted Aug 13, 2021 - 07:27 CEST
This incident affected: STO2 (Stockholm) (Block Storage).