Platform outage

Incident Report for CVaaS

Resolved

The data processing pipeline has fully recovered, and the service is now operating in a healthy state.

The ingestion of device telemetry data was not impacted during the incident. There was no data loss.

Postmortem for this incident is still pending. Our team is currently actively working on reviewing this incident, and postmortem update.
Posted Oct 12, 2025 - 14:15 UTC

Monitoring

The processing of the telemetry data backlog is progressing well. We estimate that the system will have full catch up in about 1h.
Posted Oct 12, 2025 - 13:14 UTC

Update

We have recovered the service and our system is currently processing the last 2 hours of telemetry data. The service is currently experiencing degraded performance.
Posted Oct 12, 2025 - 12:45 UTC

Update

We are continuing to work on a fix for the issue.
Posted Oct 12, 2025 - 12:11 UTC

Identified

The issue has been identified and a fix is being implemented.
Posted Oct 12, 2025 - 11:21 UTC

Investigating

We are currently investing an issue affecting the platform.
Posted Oct 12, 2025 - 11:05 UTC
This incident affected: us-central1-a (Core Platform, Network Provisioning, Events, Login, CV-CUE).