From approximately 18:00 UTC on July 23rd to 15:00 UTC on July 29th, video completion rates for all inventory were calculated incorrectly.
Scope of Impact
During the incident window, all clients using products relying on video completion rates were affected. Predicted video completion rates trended lower than actual rates, which may have impacted bidding behavior of Line Items and performance on tags.
2020-07-23 18:00: Incident Started: bad change released to video completion rate pipeline
2020-07-27 20:36: Incident Escalated
2020-07-29 15:00: Incident Resolved: corrected predicted video completion rates pushed to ad-serving platform and client notification posted
A configuration change led to video completion rates being incorrectly calculated.
Our engineers resolved the issue by correcting video completion rate tables and pushing the corrected values to our ad-serving platform. A new metric and alert was put in place to prevent a recurrence of this issue.
*Continue improving monitoring for potential issues with video completion rate calculations and other video-related metrics
The incident has been fully resolved. We apologize for the inconvenience this issue may have caused, and thank you for your continued support.
We have identified and patched the following issue:
We are monitoring our systems closely, and will provide an update as soon as the issue has been fully resolved.