Creative Batch Uploader and Batch Segment Service Unavailable
Incident Report for Xandr

Incident Summary

From approximately 15:10 to 19:10 UTC on July 15, 2020, a storage server in our NYM2 data center became unresponsive.

Scope of Impact

During the incident window, customers were not able to utilize the creative batch uploader service. Upon further investigation, this incident also resulted in customers being unable to utilize the batch segment service.

Timeline (UTC)

2020-07-15 15:10: Incident began, received alert that server was unresponsive.

2020-07-15 15:30: Engineering began troubleshooting the issue.

2020-07-15 15:45: Issue was escalated to the third party responsible for the server.

2020-07-15 19:00: Engineer on the third party team was able to restore the server and all affected services were back online.

2020-07-15 19:10: Confirmed that the issue was resolved.

Cause Analysis

The incident was caused due to a server controlled by a third party. This server was responsible for creative batch uploader as well as batch segment service which then became unavailable to our clients.

Resolution Steps

The third party responsible for this server resolved the issue. Xandr is reviewing the escalation process involving external parties for update.

Posted Aug 03, 2020 - 13:00 UTC


The incident has been fully resolved. We apologize for the inconvenience this issue may have caused, and thank you for your continued support.

Posted Jul 15, 2020 - 21:18 UTC

We have identified the following issue:

  • Component(s): Batch Segments, Creative uploads
  • Impact(s):
    • Unable to upload creatives via creative batch uploader or upload files via batch segment service
  • Severity: Minor Outage
  • Datacenter(s): NYM2

Our engineers are actively working towards a resolution, and we will provide an update as soon as possible. Thank you for your patience.

Posted Jul 15, 2020 - 20:33 UTC