(Resolved) 2018-01-02 15:25 UTC: Degraded test results processing

Last updated: Tue Jan 02 23:59:11 GMT 2018

2018-01-02 15:25 UTC: Processing of test results uploaded by cloud and enterprise agents is currently degraded. Path visualization and network end to end test results are currently affected. Data ingestion is delayed for ~10 minutes.
Our operations team is investigating the issue. We'll continue to provide updates on at least hourly basis as the events unfold towards resolution.

2018-01-02 15:55 UTC: Data ingestion delay has increased to ~20 minutes.

2018-01-02 16:00 UTC: Data ingestion delay has decreased to ~15 minutes.

2018-01-02 16:35 UTC: Data ingestion delay has increased to ~30 minutes.

2018-01-02 17:00 UTC: Network end to end test results processing degradation is now resolved. Work on resolving degradation of path visualization data processing is still in progress.

2018-01-02 23:15 UTC: Root cause was determined to be an overloaded service in our backend. The service was scaled up to accommodate the current processing needs. Test data ingestion is up to date as of now.