Ashburn Data Center Extended Maintenance Window

Resolved
Resolved

As of 20:40 UTC, all email servers are back in service and processing mail without any issues. Spam quarantine notification backlog is clearing for old emails while new emails are up to date.

Web servers in Ashburn data center are back in service as of 22:40 UTC and browsing is once again now available via both NAM data centers.

Full service is restored as quarantine backlog is fully clear.

Monitoring

As of 20:40 UTC, all email servers are back in service and processing mail without any issues. Spam quarantine notification backlog is clearing for old emails while new emails are up to date.

Web servers in Ashburn data center are back in service as of 22:40 UTC and browsing is once again now available via both NAM data centers.

Monitoring of both services continues and we will declare full service restore and provide final update once quarantine backlog is fully clear.

Update

Efforts are underway to bring email and web servers in Ashburn data center back in service. Network configuration is now in progress after installing the new switch. While we do not have an ETA available, we are expecting remaining steps to be completed soon.

As previously communicated, email and web services continue to be provided via alternate data centers. Next update will be posted once new information is available.

Identified

At 17:16 UTC, Symantec Email Quarantine (NAM Instance) has been validated and enabled back in service.

Email and Web servers in Ashburn D.C remain out of service while engineers continue on restoration steps. There is no customer impact as both services are being offered via alternate data centers.

Next update will be posted as soon as new information becomes available.

Update

Our incident management team along with networks team is working non-stop to identify the cause and in parallel restore service. Initial investigations point to a possible issue with network switch.

Symantec Email Quarantine (NAM Instance) along with Ashburn Data center remains out of service. Next update will be posted at 20:00 UTC or as soon as new information becomes available.

Investigating

Please be advised that our scheduled maintenance for Ashburn Data center is still running and as a result following items are currently impacted due to site being out of service:

North American instance of Symantec Email Quarantine is currently unavailable and users may notice a delay in receiving notifications. Track n Trace search will show partial results as servers in Ashburn D.C are out of service

Our email and web services continue to be provided via alternate data-centers without any issue. Engineering team is handling this issue as high priority and we will provide next update at 16:00 UTC.

Began at:

Affected components