Retroactive: Bidder Metrics UI Unavailable on May 26th from 15:00 - 17:00 UTC
Incident Report for Xandr
Postmortem

RETROACTIVE: Bidder Metric UI unavailable on May 26th from 15:00 – 17:00 UTC


Incident Summary

From approximately 15:00 UTC on Friday, May 26 to 17:00 UTC on Friday, May 26, 2023, users were unable to access the metrics page in the Bidder UI, during the impact window.


Incident Impact

  • Nature of Impact(s): Page load failures and errors in user interface.
  • Timeframe: ~2.00 Hours.15:00 UTC on Friday, May 26 to 17:00 UTC on Friday, May 26, 2023.
  • Scope: Global
  • Components: Bidder Metrics UI



Cause Analysis

The issue was caused during a password rotation process where the pseudouser account was disabled, as a result this led to the loss of access to the metrics page in the Bidder UI, during the impact window.

 


Resolution Steps

The issue was resolved by updating the password and synchronizing it to a permitted user account. This restored the access to the metrics page in the Bidder UI.



Follow-up Items

Inspect relevant alerts and ensure the alerts are set with better threshold parameters to avert similar occurrences in the future that would potentially result in causing a similar IM




Posted Jun 26, 2023 - 11:39 UTC

Resolved

The incident has been fully resolved.:

  • Component(s): Bidder Metrics UI
  • Impact(s):
    • Page load failures and errors in user interface
  • Not Impacted:
    • Ad Serving
    • Bidding
    • API
  • Geolocation(s): Global (Global)

We apologize for the inconvenience this issue may have caused, and thank you for your continued support.

Posted May 30, 2023 - 16:41 UTC