At this point we have satisfactorily mitigated the issues and continue to work with customers that have faced specific corner cases. If you have questions, or need further assistance please reach out to support@arista.com. Thank you!
Posted Oct 16, 2024 - 19:00 UTC
Monitoring
Hello CVaaS users,
We have successfully restored the reset event rules config for vast majority of impacted customers. We will directly reach out and assist with a few that has fallen into a corner case where automatic recovery is not possible.
Thank you for your patience and understanding in this incident. If there are any questions, or you're still seeing unexpected behaviors in event rules generation, or related config, please do not hesitate to reach out to support@arista.com.
We will continue to monitor and fully close this incident once all customers we've identified have been satisfactorily assisted.
Posted Oct 10, 2024 - 22:54 UTC
Update
Previous update made an error on affected clusters. This has been corrected. We apologize for the miscommunication.
Posted Oct 08, 2024 - 20:01 UTC
Identified
We currently have root caused why we have encountered this issue, and currently focusing on restoration. All services are operational; there is no service downtime at this time.
Posted Oct 08, 2024 - 19:26 UTC
Investigating
We've identified a catastrophic issue with a recent rollout of a new feature, that lead to unintended consequence of specific data being reset for customers, for the following features: - Customer specified event generation rules. Default event generation rules are not impacted. - Customer specified event notification/alerting timezone and customer specified URL within the event alert which points back towards CV. Event alerts will still be getting sent. - Topology Role Hints
For the features above, the reset would look to the user as if no custom rules have been set.
We are currently working on restoration of the data. We will be working directly with every customer that have been affected by this incident once we have more information.
Posted Oct 08, 2024 - 19:25 UTC
This incident affected: ausoutheast-1 (Events), na-northeast1-b (Events), uk-1 (Events), and us-central1-b (Events).