18 Dec Facebook discrepancy and Data Errors – 2016
For the third time these four months, Facebook issue a clarification regarding inaccuracies with the metrics they’ve been providing to advertisers. On December 9th FB posted in their newsroom:
“Update on December 16, 2016: We’ve uncovered an issue for a small group of Instant Articles publishers that impacts reporting in comScore. comScore alerted us to the issue, and we’ve since identified this is a result of a recent Facebook update that impacted publishers using our legacy comScore integration who support HTTPS on their websites. This caused an underreporting of iPhone traffic from Facebook in comScore products between Sep 20 to Nov 30, 2016. iPad and Android traffic were not affected. We have fixed the issue and are working with comScore to produce updated estimates for the relevant time periods for the small group of partners affected. We have reached out to affected publishers.”
Let’s look at the detail of each inaccuracy and understand it:
- Facebook reported that they’ve identified a discrepancy between the counts for the Like and Share Buttons via their Graph API and the counts when you enter a URL into the search bar in the Facebook mobile app.
In this case, the impact of the error seems relatively small – not many, if any, businesses would be looking to make comparisons in this way, so the data displayed on the mobile app is likely not going to have had a major impact on measurements.
- Facebook reported that they’ve been miscounting when Reactions occurred on Facebook Live videos. Facebook’s been providing Facebook Live creators with total Reactions counts on their content, but the data they’ve been providing was inaccurate because they were only counting one Reaction per live viewer, as listed in the “On Post” data. Users often register multiple Reactions during a video. These subsequent Reactions were being incorrectly listed in the “On Shares” count. This means that your Reactions counts would have made it seem like there were more people reacting to replays of the broadcast than there actually were – the total counts were, and are, accurate, but Facebook was mis-allocating exactly when those responses actually occurred.
- Facebook clarified that they’d been reporting their Average Duration of Video Viewed metric wrong. Which means, all those people who were watching for less than three seconds were not being counted in the results, meaning the actual average duration of viewed content was lower than what was being reported.
“We informed our partners and made sure to put a notice in the product itself so that anyone who went into their dashboard could understand our error. We have also reviewed our other video metrics on the dashboard and have found that this has no impact on video numbers we have shared in the past, such as time spent watching video or the number of video views. We want our clients to know that this miscalculation has not and will not going forward have an impact on billing or how media mix models value their Facebook video investments.”
- Facebook reported that it had uncovered a bug which meant that one of the dashboards on the Page Insights tab was displaying incorrect data. Facebook says the reach on this tab was miscalculated, showing a sum of daily reach but not removing repeat visitors from the count.
- Facebook found an error in their reporting of average time spent per article on Instant Articles, with a calculation mistake leading to an over-reporting of this stat by around 7%-8%. Facebook said that they were incorrectly counting the “histogram of time spent”, instead of reflecting the total time spent reading an article divided by total views.
- Facebook found an error in their reporting of referrals from apps. This data’s supposed to be a reflection of clicks that went directly to an app or website, but Facebook found that they were also counting other clicks on those posts, including clicks to view photos or video.
- Facebook confirmed that comScore had identified an error with Instant Articles data which caused an underreporting of iPhone traffic for IA content between September 20th to November 30th, 2016. According to The Wall Street Journal, the error impacted less than 1% of the traffic for affected publishers, although, comScore estimates that some publishers’ traffic was undercounted by 10% to 20% in the period.