From approximately 09:56 UTC to 15:00 UTC on on September 14, 2021, users were intermittently unable to view order home monitoring screens.
During the incident window, customers were intermittently unable to view order home monitoring screens and therefore unable to monitor the performance of their line items, particularly GDALIs.
2021-09-14 09:56: Incident started
2021-09-14 14:29: Incident escalated to engineering
2021-09-14 14:50: Engineering initiates mitigation steps
2021-09-14 15:00: Incident Resolved
The incident was caused by an increase in traffic during EU business hours.
Our engineers resolved the issue by moving the order home data pods to a different data center.
Review and add additional alerting to the backend systems controlling the order home monitoring screens.
Implement more monitoring to this area to help identify potential issues prior to becoming an outage.
The incident has been fully resolved. We apologize for the inconvenience this issue may have caused, and thank you for your continued support.
We have patched the issue and are monitoring our systems closely. We will provide an update as soon as the issue has been fully resolved.
We are currently investigating the following issue:
We will provide an update as soon as more information is available. Thank you for your patience.