Some clients experiencing drop in impressions for PSP demand partner Adform
Incident Report for Xandr
Postmortem

Incident Summary

From approximately November 19th, 2021 21:00 UTC to November 21st, 2021 01:00 UTC, customers experienced a massive drop in impressions from the Prebid Server Premium (PSP) demand partner Adform. 

Scope of Impact

During the incident window, customers experienced a downtime in requests/delivery from the PSP demand partner Adform, some customers were also unable to save/edit existing PSP configs containing Adform.

Timeline (UTC)

  • 2021-11-19 21:00: Incident started 
  • 2021-11-20 11:00: Incident escalated to engineering.
  • 2021-11-21 01:00: Incident Resolved

Cause Analysis

The incident intialized when Adform, a Prebid Server Adapter/PSP demand partner, removed support for several optional parameters in their adapter. We made an update to reflect this change, however, this did not automatically update the existing configs containing the optional parameters. 

Resolution Steps

Bulk edit of the new Adform parameters from integer to string resolved the issue affecting the Prebid server premium calls and re-uploading all of the Adform parameters resolved the remaining.

**
Next Steps**

* Add validation to the bulk edit process to ensure expectation is matched

Posted Dec 23, 2021 - 16:52 UTC

Resolved

The incident has been fully resolved. We apologize for the inconvenience this issue may have caused, and thank you for your continued support.

Posted Nov 20, 2021 - 02:02 UTC
Monitoring

We have patched the issue and are monitoring our systems closely. We will provide an update as soon as the issue has been fully resolved.

Posted Nov 20, 2021 - 00:13 UTC
Identified

We have identified the cause of the issue, and our engineers are actively working towards a resolution. We will provide an update as soon as possible. Thank you for your patience.

Posted Nov 19, 2021 - 14:02 UTC
Investigating

We are currently investigating the following issue:

  • Component(s): Ad Serving, Prebid Server Premium
  • Impact(s):
    • Decrease in requests sent to Bidder customers
  • Severity: Minor Outage
  • Datacenter(s): Global

We will provide an update as soon as more information is available. Thank you for your patience.

Posted Nov 19, 2021 - 12:51 UTC