Incident Summary
Missing up to date data in some reports - reporting data was available but only up to a particular point in the past. That particular point for each report was different.
Scope of Impact
During 2020-09-17 to 2020-09-21, the following reports contained contiguous, yet not up to date data: Platform Seller Analytics, Prebid Analytics, YX Report, Platform Video Analytics, Video Analytics, Mobile Carrier Analytics, KV Analytics, Platform Domain Detection, Platform Inventory Analytics, Client error reports.
Timeline (UTC)
2020-09-17 15:00 : Incident Started.
2020-09-18 07:42 : Incident was reported. initially seen via “Seller Bid Error Report” missing data.
2020-09-18 14:32 : Issue resolved.
2020-09-18 13:59 : Our internal team decided to reopen the issue, as a part of a broader database fix.
2020-09-21 16:00 : Issue resolved.
Cause Analysis
Some of the data syncing jobs failed, which occasionally occur, and no alert was generated as expected. This caused a delay in the issue resolution.
Resolution Steps
The jobs were re-run to restore the missing data.
Next Steps
We are working on an improved model for dealing with operational support issues.
The incident has been fully resolved. We apologize for the inconvenience this issue may have caused, and thank you for your continued support.
We are currently investigating the following issue:
We will provide an update as soon as more information is available. Thank you for your patience.