From approximately 08:12 UTC to 08:56 UTC on Tuesday, October 22, Console Bidder was temporarily unavailable for requests nearest our European data centres (Amsterdam and Frankfurt).
During the incident window, customers may have seen an increase in defaults and PSAs.
2019-10-22 08:12: Incident started, Console Bidder servers begin crashing
2019-10-22 08:55: Incident escalated to engineering
2019-10-22 09:05: Incident resolved
Console Bidder received requests for a new placement that had not yet made it into memory. The missing object triggered a latent error causing Console Bidder instances to crash on roadblock evaluation.
The issue resolved automatically when servers restarted because the placement became available in memory.
Fix latent null object bug
The following incident has been fully resolved, and we will post a post-mortem as soon as we have completed one:
We apologize for the inconvenience this issue may have caused, and thank you for your continued support.