From approximately 21:00 UTC on September 13 to 00:58 UTC on September 14, 2021 the LAX1 datacenter suffered from a high rate of timeout issues.
During the incident window some customers using console bidders in LAX1 may have observed a drop in delivery.
2021-09-13 21:00: Incident started
2021-09-13 23:45: Escalated to engineering
2021-09-14 00:58: Incident resolved
Engineering introduced a defect which, in addition to connection limitations, overstressed the LAX1 datacenter and caused bidder timeouts.
A reconfiguring of the connections within the datacenter resolved the timeout issue. Engineering has since identified and fixed the root defect.
Review and update internal alerting process to alert engineering of similar issues sooner.
The following incident has been fully resolved, and we will post a post-mortem as soon as we have completed one:
We apologize for the inconvenience this issue may have caused, and thank you for your continued support.