VPS (multiple) Network Interruption

  •   Posted: 7:14 pm EST 12/25/15

  •   Updated: 8:05 pm EST 01/05/16

Disruption Start Time: 7:10 pm EST 12/25/15

Estimated duration:

Status: Resolved

We are getting in touch to provide you with a Hosting Status update. As of 19:10 EST a network issue was identified with some VPS servers. Our System Administration team is actively working on this now and we will update this post as more information becomes available. Websites hosted and other services on this server may be inaccessible until this is resolved.

We sincerely apologize for the inconvenience this issue has caused. We take uptime extremely seriously and know it is absolutely vital to our customer's web presence. If you have any further questions please let us know and we will do our best to answer them!

Please Note: This status post has been replaced. Please see https://status.asmallorange.com/?id=596 for current news.

UPDATE: As of 20:15 EST, our System Administrators have identified the cause and are working diligently on the fix. We will be updating this thread with the latest status of the issue. Many apologies, your uptime is extremely important to us.

UPDATE: As of 21:00 EST, our System Administrators are still working on bringing the VPS servers back up. During this work, Cloud Commands will remain deactivated to ensure a quick resolution to the existing issues. We highly appreciate your patience in this matter.

UPDATE: As of 22:00 EST, our System Administrators are still working on bringing the VPS servers back up. We will be posting latest status updates in this thread. We appreciate your patience in this matter.

UPDATE: As of 00:05 EST, our System Administrators continue to work on this.

UPDATE (2:20 AM EST): Our team continually works on this issue. Unfortunately, we do not have an ETA yet but we believe that the service should be restored soon.

UPDATE (7:40AM EST): At this time, the majority of affected services have been restored and should be back online and serving now. There remains a small portion of VPS services offline and we expect that service for that population to be restored as soon as possible.

UPDATE (10:20AM EST): Additional part of affected VPS instances has been brought online. We are still working on the rest of VPSes and should get them back up soon.

UPDATE (13:40 PM EST 12/26): Due to a resurgence of issues, a number of VMs are still being affected by this ongoing service interruption. Our Technology team is working to resolve the issues as soon as is possible.

UPDATE (2:50 PM EST 12/26): Affected VPS instances are performing reboots, which is bringing them back online. Access to OnApp is still disabled, in order to assure as fast and stable reboot cycle as possible.

UPDATE (5:20 PM EST 12/26): We are still working to resolve issues with VPS's that are still offline. Our most senior staff is working on the problem at hand. We appreciate your patience.

UPDATE 19:55 We are very glad to report that this issue has been resolved. Please note that individual VPS's may require file systems checks to be brought online. If you continue to have any further trouble with your VPS, please do not hesitate to reach out to one of our Tech Support Ninjas.

UPDATE: 23:55 12/26 We are investigating further problems that are being reported. We are terribly sorry for the unfortunate duration of these problems. We are taking this outage very seriously and will work tirelessly until it is fully resolved.

UPDATE: 01:40 12/27 We believe that the majority of VPS's have been fixed and are back online. We are running additional probes to determine which VPS's may be still need manual intervention to be returned to an online status, and taking steps to resolve them.

UPDATE: 04:38 12/27 Majority of the machines are now up, we're currently scanning for any remaining down virtual machines and will be correcting them as soon as possible.

UPDATE (12:20 PM EST 12/27): Due to another resurgence of issues, a number of VMs are still being affected by this ongoing outage. Our Technology team is working to resolve the issues as soon as is possible.

UPDATE (1:00 PM EST 12/27): Due to a resurgence of issues, a number of VMs are still being affected with the service disturbance. Our team is still working on getting affected instances back online. Unfortunately, we do not have any ETA available at the moment.

UPDATE (3:50PM EST 12/27): A part of affected VPS instances has been brought back online. The rest of instances is still being scanned for the read only state and we are running fsck on them to get the file system in a writable healthy state and bring all services back online.

UPDATE (6:35PM EST 12/27) VPS instances that are still affected with the service disturbance are queued for a reboot which should bring them back online. It will be followed by an additional scan that will catch all the instances that may still experience problems. Thank you for your patience during this time. We recognize that extended downtime such as this is very difficult.

UPDATE (8:45 PM EST 12/27) We are aware of continued issues with some VPS instances. If your VPS is down, you can be assured that we are working round the clock to resolve these issues and restore all services to a fully functional status. Thank you once again for your extreme patience, and we are terribly sorry for the length of the outage.

UPDATE (12:30 AM EST 12/28) We wanted to update you regarding the Cloud issues we are experiencing. We are still working to resolve problems permanently, and are aware of some stability issues still occurring. If your VPS is still offline, please know that we are aware and are very concerned with getting it back online as soon as possible. We are genuinely grateful for your patience through this.

UPDATE: (02:40AM EST 12/28) We are aware of continued issues with some VPS instances. If your VPS is down, you can be assured that we are working round the clock to resolve these issues and restore all services to a fully functional status. Thank you once again for your extreme patience, and we are terribly sorry for the length of the outage.

UPDATE: (04:40AM EST 12/28) Our team continually works on this issue. Unfortunately, we do not have an ETA yet.

UPDATE: (06:40AM EST 12/28) We are aware of continued issues with some VPS instances. If your VPS is down, you can be assured that we are working round the clock to resolve these issues and restore all services to a fully functional status. Thank you once again for your extreme patience, and we are terribly sorry for the length of the outage.

UPDATE: (09:00AM EST 12/28) All affected VPS instances are still being worked on and we are constantly running fixes to get them back online. Our system administration team is working on the issue around the clock, and bring instances back up.

UPDATE: (11:00AM EST 12/28) We are running a new round of reboots and fixes on affected VPS instances. Unfortunately, we do not have any ETA available at the moment, but we expect some improvement soon. Thank you for your patience