Data ingestion processing lag

Incident Report for CVaaS

Postmortem

A reconfiguration of the proxy layer forced all the clients to reconnect at the same time, adding pressure on our core platform. The pressure caused data to accumulate in our ingestion queue, resulting in processing delay.

Posted Jul 02, 2024 - 10:52 UTC

Resolved

This incident has been resolved.
Posted Jul 02, 2024 - 10:44 UTC

Monitoring

The processing lag is currently recovering.
Posted Jul 02, 2024 - 10:40 UTC

Investigating

We are currently investigating an increase in processing lag.
Posted Jul 02, 2024 - 10:12 UTC
This incident affected: us-central1-c (Core Platform).