VMware Cloud on AWS and VMware Cloud on PKS: Availability Issue

Resolved
Resolved

VMware Cloud PKS and VMware Cloud on AWS availability issues in US-WEST-2, EU-WEST-1, and US-EAST-1 stands resolved and fully recovered. Engineering Team has restored propagation for Route 53 Private DNS changes. This is the final notification for this incident.

Start Time: May 10, 2019, 8:50 PM UTC End Time: May 11, 2019, 9:46 AM UTC

Impact: None

Update

VMware Cloud PKS and VMware Cloud on AWS continues to experience availability issues in US-WEST-2, EU-WEST-1, and US-EAST-1. Engineering Team has restored propagation for Route 53 Private DNS changes. Teams continue to work towards full recovery. Queries to existing DNS records are not affected by this issue. Next update in 1 hour.

Start Time: May 10, 2019 8:50 PM UTC End Time: N/A

Impact: Cluster creation and deletion operations may fail. Customers can continue to use their existing clusters. Queries to existing DNS records are not affected by this issue.

Update

VMware Cloud PKS and VMware Cloud on AWS continues to experience availability issues in US-WEST-2, EU-WEST-1, and US-EAST-1. Engineering Team continues to make progress towards restoring propagation for Private DNS. A majority of hosts have processed all updates, and the team is working to accelerate recovery of the remaining hosts. Queries to existing DNS records are not affected by this issue. Next update in 1 hour.

Start Time: May 10, 2019 8:50 PM UTC End Time: N/A

Impact: Cluster creation and deletion operations may fail. Customers can continue to use their existing clusters. Queries to existing DNS records are not affected by this issue.

Update

VMware Cloud PKS and VMware Cloud on AWS continues to experience availability issues in US-WEST-2, EU-WEST-1, and US-EAST-1. Engineering Team processed the majority of the backlog of queued changes for Public DNS. Private DNS changes are still experiencing propagation delays. We continue to work towards full recovery. Queries to existing DNS records are not affected by this issue. Next update in 1 hour.

Start Time: May 10, 2019 8:50 PM UTC End Time: N/A

Cluster creation and deletion operations may fail. Customers can continue to use their existing clusters. Queries to existing DNS records are not affected by this issue.

Update

VMware Cloud PKS and VMware Cloud on AWS continues to experience availability issues in US-WEST-2, EU-WEST-1, and US-EAST-1. Engineering Team processed the majority of the backlog of queued changes for Public DNS. Private DNS changes are still experiencing propagation delays. We continue to work towards full recovery. Queries to existing DNS records are not affected by this issue. Next update in 1 hour.

Start Time: May 10, 2019 8:50 PM UTC End Time: N/A

Impact: Cluster creation and deletion operations may fail. Customers can continue to use their existing clusters. Queries to existing DNS records are not affected by this issue.

Update

VMware Cloud PKS and VMware Cloud on AWS continues to experience availability issues in US-WEST-2, EU-WEST-1, and US-EAST-1. Engineering Team is observing steady recovery on the propagation times of DNS edits to public and private hosted zones, with the oldest changes propagating first. Teams continuing to work through the backlog of queued changes. Queries to existing DNS records are not affected by this issue.Next update in 1 hour.

Start Time: May 10, 2019 8:50 PM UTC End Time: N/A

Impact: Cluster creation and deletion operations may fail. Customers can continue to use their existing clusters. Queries to existing DNS records are not affected by this issue.

Update

VMware Cloud PKS continues to experience availability issues in US-WEST-2, EU-WEST-1, and US-EAST-1. Engineering Team is working on the problem with the increased propagation times of DNS edits to public and private hosted zones, and increased failures for Delete Hosted Zone, Associate Private Hosted Zone and Disassociate Private Hosted Zone API and Console operations. Queries to existing DNS records are not affected by this issue. Next update in 1 hour.

Start Time: May 10, 2019 8:50 PM UTC End Time: N/A

Impact: Cluster creation and deletion operations may fail. Customers can continue to use their existing clusters. Queries to existing DNS records are not affected by this issue.

Update

VMware Cloud PKS continues to experience availability issues in US-WEST-2, EU-WEST-1, and US-EAST-1. Engineering teams have identified a possible root cause and are progressing toward a solution to restore normal service.

Start Time: May 10, 2019 8:50 PM UTC End Time: N/A

Impact: Cluster creation and deletion operations may fail. Customers can continue to use their existing clusters.

Identified

We are continuing to investigate the situation and will provide more updates as we have more information.

Start Time: May 10, 2019 8:50 PM UTC End Time: N/A

Impact: Cluster creation and deletion operations may fail. Customers can continue to use their existing clusters.

Investigating

VMware Cloud PKS is experiencing an availability issue in US-WEST-2, EU-WEST-1, US-EAST1. Engineering teams are engaged to resolve the issue and restore service.

Start Time: May 10, 2019 8:50 PM UTC End Time: N/A

Impact: Cluster creation and deletion operations may fail. Customers can continue to use their existing clusters.

Began at:

Affected components