Incident Summary
From 16:00 UTC on Thursday, March 19th to 00:05 on Saturday, March 21st an impression bus release caused our system to erroneously expect incorrect contract terms for Prebid demand partners, causing Prebid Server Premium demand to be rejected.
Scope of Impact
During the incident window, the Prebid Server Premium product was down and Prebid Server Premium demand was rejected.
Timeline (UTC)
2020-03-19 06:00: Incident Started: Impression Bus release introduced the issue
2020-03-20 14:00: Prebid Server Premium clients saw a drop in impression volume
2020-03-20 15:43: Impression Bus release was identified as the root cause
2020-03-20 16:52: Incident ticket created
2020-03-21 00:05: Incident Resolved: Impression Bus release was completed to address issue introduced by previous release
Cause Analysis
The root cause of the incident was due to an Impression Bus release introduced an erroneous assumption on contract rate cards for Prebid demand partner buyer seats. None of the Prebid demand partner buyer seats had the necessary contract rate cards set up. This logic was unresolvable without an Impression Bus release.
Resolution Steps
Our engineering team released a hot fix to Impression Bus to resolve the issue from the previous release.
Next Steps
The incident has been fully resolved. We apologize for the inconvenience this issue may have caused, and thank you for your continued support.
We have identified the following issue:
Our engineers are actively working towards a resolution, and we will provide an update as soon as possible. Thank you for your patience.